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Abstract 

Stochastic optimal control problems governed by delay equations with delay in the control are 
usually more difficult to study than the the ones when the delay appears only in the state. This is 
particularly true when we look at the associated Hamilton-Jacobi-Bellman (HJB) equation. Indeed, 
even in the simplified setting (introduced first by Vinter and Kwong [44] for the deterministic case) 
the HJB equation is an infinite dimensional second order semilinear Partial Differential Equation 
(PDE) that does not satisfy the so-called “structure condition” which substantially means that “the 
noise enters the system with the control”. The absence of such condition, together with the lack 
of smoothing properties which is a common feature of problems with delay, prevents the use of the 
known techniques (based on Backward Stochastic Differential Equations (BSDEs) or on the smoothing 
properties of the linear part) to prove the existence of regular solutions of this HJB equation and so 
no results on this direction have been proved till now. 

In this paper we provide a result on existence of regular solutions of such kind of HJB equations 
and we use it to solve completely the corresponding control problem finding optimal feedback controls 
also in the more difficult case of pointwise delay. The main tool used is a partial smoothing property 
that we prove for the transition semigroup associated to the uncontrolled problem. Such results holds 
for a specific class of equations and data which arises naturally in many applied problems. 


Contents 

1 Introduction 

1.1 Our results in a simple motivating case. 

1.2 Plan of the paper. 

2 Preliminaries 

2.1 Notation. 

2.2 C-derivatives. 

3 Setting of the problem and main assumptions 

3.1 State equation . 

3.2 Infinite dimensional reformulation . 

3.3 Optimal Control problem. 

4 Partial smoothing for the Ornstein-Uhlenbeck semigroup 

4.1 Partial smoothing in a general setting. 


2 

2 

i 

6 

6 

6 


8 

8 




1 ^ 

Hi 


1 

















4.1.1 Second derivatives. 

4.2 Partial smoothing in our model. 

4.2.1 C = I . 

4.2.2 C = B . 

5 Smoothing properties of the convolution 

6 Regular solutions of the HJB equation 

6.1 Existence and uniqueness of mild solutions 

6.2 Second derivative of mild solutions .... 


15 

16 

17 

18 


1 ^ 

24 

27 


7 Verification Theorem and Optimal Feedbacks 

7.1 /C-convergence and approximations of solutions of the HJB equation 

7.2 The Fundamental Identity and the Verification Theorem. 

7.3 Optimal feedback controls and v = V . 


1 ^ 

29 

35 


1 Introduction 

Optimal control problems governed by delay equations with delay in the control are usually harder to 
study than the ones when the delay appears only in the state (see e.g. [3J Chapter 4] and [IHllSO])- This 
is true already in the deterministic case but things get worse in the stochastic case. When one tries 
to apply the dynamic programming method the main difficulty is the fact that, even in the simplified 
setting introduced first by Vinter and Kwong [44] in the deterministic case (see e.g. [2^ for the stochastic 
case), the associated HJB equation is an infinite dimensional second order semilinear PDE that does not 
satisfy the so-called “structure condition”, which substantially means that the noise affects the system 
only “through the control”. 

The absence of such condition, together with the lack of smoothing properties which is a common 
feature of problems with delay, prevents the use of the known techniques, based on BSDE’s (see e.g. [23]) 
or on fixed point theorems in spaces of continuous functions (see e.g. |5lil[T3l|27l|28]) or in Gauss-Sobolev 
spaces (see e.g. [iniiisj), to prove the existence of regular solutions of this HJB equation: hence no results 
in this direction have been proved till now. The viscosity solution technique can still be used (see e.g. 
[30]) but to prove existence (and possibly uniqueness) of solutions that are merely continuous. This is an 
important drawback in this context, since, to prove the existence of optimal feedback control strategies 
through the dynamic programming approach, one needs at least the differentiability of the solution in 
the “space-like” variable. 

The main aim of this paper is twofold: first to provide a new result of existence of regular solutions 
of such HJB equations that holds when the state equation depends linearly on the history of the control 
and when the cost functional does not depend on such history; second to exploit such result to solve 
the corresponding stochastic optimal control problem finding optimal feedback control strategies. This 
allows to treat satisfactorily a specific class of state equations and data which arises naturally in many 
applied problems (see e.g. iniiiij iMiisniiMlij). Note that, differently to what often happens in the 
literature (see e.g. mi 113130]) here we are able to treat also the case of “pointwise delay” which gives 
rise to an unbounded control operator in the state equation. 

The key tool to prove such results is the proof of a “partial” smoothing property for the transition 
semigroup associated to the uncontrolled equation which we think is interesting in itself and is presented 
in Section 3. 

We believe that such tool may allow to treat also examples where the state equation depends on the 
history of the state variable, too. To keep things simpler, here we choose to develop and present the 
result when this does not happen leaving the extension to a subsequent paper. 


1.1 Our results in a simple motivating case 

To be more clear we now briefly describe our setting and our main result in a special case. Let (fl, J^, P) be 
a complete probability space and consider the following linear controlled Stochastic Differential Equation 
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(SDE) in K: 


( dy{s) = aoy{s)dt + bou{s)ds + J^^bi{^)u{s + + adWs, sG[i,r] 

S yit) = yo, (1-1) 

[ = uo(0, C e [-d,o). 

Here VH is a standard Brownian motion in R, and {J^t)t>o is the augmented filtration generated by W. We 
assume oq, &o S R, cr > 0. The parameter d > 0 represents the maximum delay the control takes to affect 
the system while &i is the density function taking account the aftereffect of the control on the system. 
The easier case is when bi G T^([—(i,0],R) (“distributed delay”) while a more difficult, yet common, case 
is when bi is a measure, e.g. a Dirac delta in —d (“pointwise delay”): both will be treated here. 

The initial data are the initial state t/o a-nd the past history uq of the control. The control u belongs 
to X [0,T], U), the space of predictable square integrable processes with values in [/ C R, closed. 

Such kind of equations is used e.g. to model the effect of advertising on the sales of a product (see 
e.g. [MllSO]), the effect of investments on growth (see e.g. |17j in a deterministic case), or, in a more 
general setting, to model optimal portfolio problems with execution delay, (see e.g. [4]) or to model the 
interaction of drugs with tumor cells (see e.g. [33] p.l7 in the deterministic case). 

In many applied cases (like the ones quoted above) the goal of the problem is to minimize the following 
objective functional 


J{t,xo,uo]u{-)) =E (4(s,?/(s))+£i(u(s))) ds + E(j){y{T)). 


( 1 . 2 ) 


where £o : [0, T] x R —>■ R, .^i : t/ —>■ R and 0 : R —R are continuous functions satisfying suitable 
assumptions that will be introduced in Section |3| It is important to note that here £o, £i and (j) do not 
depend on the past of the state and/or control. This is a very common feature of such applied problems. 

A standard wa>0 to approach these delayed control problems, introduced in [33] for the deterministic 
case (see [33] for the stochastic case) is to reformulate the above linear delay equation as a linear SDE in 
the Hilbert space "H := R x T^([—d, 0],R), with state variable Y = {Yo,Yi) as follows. 

J dY{s) = AY{s)ds + Bu{s)ds + GdWs, s G [t,T] , , 

\ Y{t) =x = {xo,xi), 

where A generates a Co-semigroup (see next section for precise definitions) while, at least formallyH 

i? : R —> 'H, Bu = {bou, bi{-)u), u G R, G : R —>■ "H, Gx = {<tx, 0), x G R. (1-4) 

Moreover xg = yo while xi(^) = ^ 6i(<r)uo(<r — (C ^ [—d, 0]) i.e. the infinite dimensional datum xi 

depends on the past of the control. 

The value function is defined as 


V(t, x) := inf E 

u(-)eL^(nx[o,T],u) 


[^(s,4b(s)) +^i(m(s))] ds -k (j){Yg(T)) 


The associated HJB equation (whose candidate solution is the value function) is 

= ^Tr GG*V^v{t,x) + {Ax,Vv{t,x))^ + Hmin{'^v{t,x)) +£o{t,x), t G [0,T], x gH, 


v{T,x) = (/>(xo), 
where, defining the current value Hamiltonian Hcv as 


(1.5) 


Hcv{p] u) := {p, Bu)^ -k £i{u) = {B*p, u)^ + £i{u) 

^It must be noted that, under suitable restrictions on the data, one can treat optimal control problems with delay in the 
control also by a direct approach without transforming them in infinite dimensional problems. However in the stochastic 
case such direct approach seems limited to a very special class of cases (see e.g. m) which does not include our model and 
models commonly used in applications like the ones just quoted. 

^Note that, when bi is a measure, the above operator B is not bounded (see next section for the precise setting). 
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we have 


( 1 . 6 ) 


Hmin{p) '■= inf Hcv{p\u). 
u£U 

The ultimate goal in studying this HJB equation is to hnd a solution v with enough regularity to prove 
a verification theorem (i.e. that v = V plus a sufficient condition for optimality) and to find an optimal 
feedback map. 

It is well known, see e.g. [3S] Section 5.5.1, that, if the value function V is smooth enough, and if the 
current value Hamiltonian JIcv always admits at least a minimum point, a natural candidate optimal 
feedback map is given by (t,x) u*{t,x) where u*(t,x) satisfies 

(VH(t, x), Bu*it, x))„ + x)) = Hrmu{VV{t, x)). 

i.e. where u*(t, x) is a minimum point of the function u i—> Hcvi^vit, x); u), K —> M. To take account of 
the possible unboundedness of B it is convenient to define, for z G K, 

Hmin{z) ■■= inf {(z,u)g + £i(-«)} =: inf Hcv{z;u)} (1.7) 

uGU uGU 

so that 

Hminip) = inf Hcv{p;u). = inf {(H*p, u)]g+ Zi(u)} =: Hmin{B*p). (1.8) 

u^U uGU 

Since u*{t, x) is a minimum point of the function u i—>■ Hcv{B*'Vv(t, x); tt), R —R, the minimal regularity 
required to give sense to such term is the existence of B*Vv{t,x) which we will call V^v{t,x) according 
with the definition and notation used e.g. in |251136) in the case of bounded B and generalized here in 
Subsection 12.21 to the case of possibly unbounded B. From now on we will use Hcv and Hmin in place 
of Hcv and Hmin writing iJmm(V^x(f, x)) in place of Hmin{'^v{t,x)) in (fTS]) . 

As announced, the main results of this paper are exactly to prove the existence of a solution of the HJB 
equation 0131) with such regularity (Section 5) and then to prove a verification theorem and the existence 
of optimal feedback controls (Section 6). Such results are difficult to get since the known techniques do 
not apply to this problem. We explain why. 

To prove the existence and uniqueness of regular solutions of HJB equation^ like (11.511 in the simplest 
case when H is Lipschitz continuous there are two main approaches in the literature: a BSDEs based 
approach and a fixed point approach. 

The BSDEs approach (see e.g. the paper [53], which is the infinite dimensional extension of results 
in [39]) is based on representing the solution v using a suitable BSDE. The regularity of v is inherited 
from the data so all the coefficients, included the Hamiltonian i7, are assumed to be at least Gateaux 
differentiable. To apply such approach the controlled state equation must satisfy the “structure condition” 
which, in our case, would mean that ImH C ImG. Our HJB equation (II.5p is out of such literature since 
the structure condition does not hold here. This is an intrinsic feature of such problem since the fact 
that Im B is not contained in Im G depends on the presence of the delay in the control. If the delay in 
the control disappears, then the structure condition hold, even if delay in the state is present (see e.g. 

[MIEQlEllEH]). 

The fixed point approach consists in rewriting ([13]) in a suitable integral form as follows. 

v{t,x) = RT-t[$\{xo) + Rs-t[Hmin(y^v{sc)) +h{s,-)]{x) ds, t G [0,r], X G'H, (1.9) 

where hy Rt, 0 < t < T we denote the transition semigroup associated to the uncontrolled version of 
equation (11.311 . In the literature such equation is treated by looking for a fixed point in suitable spaces of 
functions (at least differentiable, in some sense, in x) using, as a key assumption, a smoothing property of 
the transition semigroup Rt- i.e. that Rt transforms any bounded measurable (or continuous) function ip 
into a differentiable one with an integrable singularity of Rt^p when t ^ 0^. In this direction we mention 
the papers [SEinilljE] which treat the problem in spaces of continuous functions and the papers [101126] 
that work in spaces with respect to a suitably defined invariant measure for the uncontrolled system. In 
both cases, to get the key smoothing assumption on Rt described just above, the authors need to assume 
strong properties of the data (a null controllability assumption when working in spaces of continuous 

^Sometimes such equations are called semilinear Kolmogorov equations, see e.g. m 
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functions and again the structure condition when the live in L^) that cannot be verified in the case under 
study. 

It may be possible to attack such HJB equations with the viscosity solution approach (see e.g. [30] for 
an existence result in a similar case and m for a survey on the subject). However this approach would 
not give us any regularity. In the deterministic case there are papers miooiiii] where, as it is done in 
finite dimension (see [HIT]), a sort of regularity for the viscosity solution is proved through convexity 
methods. However such approach does not seem to work, up to now, in the stochastic case. 

In the present paper, in the spirit of the fixed point approach, we start from the observation that, 
even if the semigroup Rt in this case does not possess the above required smoothing property, it still 
regularizes some classes of functions in certain directions. In contrast to the standard smoothing property 
we call this last one partial smoothing. More precisely, in the motivating case just described, we are able 
to prove that for every bounded function / : 'H —R, of the form 

/ (xo, xi) = f(xo), for some bounded function / : R —>• R, 

the function Rt[f] is continuous, and for every x GR the derivative Rt[f]{x) exists and blows up like 
^/t even if B is unbounded (see Section H) Theorem 14.II for the general case and ProDOsition l4.9l and l4.lll 
for the application to our specific case). 

With this result at hand we act as in the fixed point approach, looking for solutions satisfying the 
integral equality ()1.9I1 . In the definition of the cost functional ^ and Iq depend on the state variable, 
which means that, in the abstract formulation, they depend only on the first component, so the partial 
smoothing property just stated fits to them. Then we can carefully apply a fixed point technique (in 
doing which the choice of the spaces must be done in a subtle way) to hnd a unique solution v to the 
integral form (11.91) of the HJB equation (Section |6l Theorem 16.31) . 

On this solution v (which we call mild solution according to most of the literature) we require the 
minimal regularity such that formula (11.91) makes sense. Such minimal regularity is not enough to prove 
a verification theorem (which needs to apply Ito’s formula to v). However we are able to prove that v 
is the limit of classical solutions of approximating equations to which Ito’s formula applies (Section 7.1, 
Lemma rT3l) . This is enough to prove the verihcation theorem (Section 7.2, Theorem 17.91) . 

Finally adding some further regularity assumptions we prove that v = V and we characterize the 
optimal control by a feedback law, which will be of the form 

u(s) = 7 (V^H(s,y(s))) , P-a.s. for almost every s G [t, T], (1-10) 

for a suitable function 7 (Section 7.3, Theorem l7.13l) . 

Notice that in the present paper we deal with a finite dimensional control delayed equation dm, 
that here in the introduction we have presented in dimension one for the sake of simplicity. The same 
arguments apply if we consider the case of a controlled stochastic differential equation in an infinite 
dimensional Hilbert space Rq with delay in the control as follows. 

( dy{t) = Aoy{t)dt + Bou{t)dt + J°^Bi{^)u{t + + adWt, t G [0,T] 

{ 2 /( 0 ) = 2 / 0 , (1-11) 

[ m(^) = uo(C), C e [-d,0). 

Here IF is a cylindrical Wiener process in another Hilbert space S, Aq is the generator of a strongly 
continuous semigroup in Rg, a G CCE^Rq), and we have to assume some smoothing properties for the 
Ornstein Uhlenbeck transition semigroup with drift term given by Aq and diffusion equal to cr, see 
Remarks 13.2114.71 and 14.121 for more details. 

1.2 Plan of the paper 

The plan of the paper is the following: 

• in Section |3| we give some notations and we present the problem and the main assumptions; 

• in Section |4| we prove the partial smoothing property for the Ornstein-Uhlenbeck transition semi¬ 
group, and we explain how to adapt it to an infinite dimensional setting; 
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• in Section [S] we introduce some spaces of functions where we will perform the fixed point argument 
and we prove regularity of some convolutions type integrals; 

• in Section [5] we solve the HJB equation (11.51) in mild sense; 

• Sectionals devoted to solve the optimal control problem. In Subsection 7.1 we approximate the 
mild solution of the HJB equation, in Subsection 7.2 we prove a verification theorem, and finally in 
Subsection 7.3 we identify the value function of the control problem with the solution of the HJB 
equation and we characterize the optimal control by a feedback law. 

2 Preliminaries 

2.1 Notation 

Let il be a Hilbert space. The norm of an element x in H will be denoted by \x\j^ or simply |a;|, if no 
confusion is possible, and by simply by (•,•) we denote the scalar product in H. We denote 

by H* the dual space of H. If K is another Hilbert space, £{H, K) denotes the space of bounded linear 
operators from H to K endowed with the usual operator norm. All Hilbert spaces are assumed to be real 
and separable. 

In what follows we will often meet inverses of operators which are not one-to-one. Let Q € C(H,K). 
Then Hq = kerQ is a closed subspace of H. Let Hi be the orthogonal complement of Hq in H: Hi is 
closed, too. Denote by Qi the restriction of Q to Hi : Qi is one-to-one and ImQi = ImQ. For k € ImQ, 
we define Q~^ hy setting 

g-i(fc) :=gr'W- 

The operator Q~^ : ImQ —>• il is called the pseudoinverse of Q. Q~^ is linear and closed but in general 
not continuous. Note that if k £ Im Q, then (k) is the unique element of {h : Q (h) = k} with minimal 
norm (see e.g. [46], p.209). 

In the following, by (H, J",P) we denote a complete probability space, and by L|,(H x [0,T],H) the 
Hilbert space of all predictable processes (^t)te[o,T] with values in H, normed by ||^||^2 (nx[ 0 T] h) ~ 

EJo\Zt?dt. 

Next we introduce some spaces of functions. We let H and K be Hilbert spaces. By Bi,{H,K) 
(respectively Cb{H,K), UCb{H,K)) we denote the space of all functions f : H ^ K which are Borel 
measurable and bounded (respectively continuous and bounded, uniformly continuous and bounded). 

Given an interval / C M we denote by C{IxH, K) (respectively Cb{I x i7, K)) the space of all functions 
f : I X H ^ K which are continuous (respectively continuous and bounded). C'°’^(/ x H,K) is the space 
of functions / £ C{I x H) such that for alH S / f(t, •) is Frechet differentiable. By {7C'^’^(/ x H, K) we 
denote the linear space of the mappings f : I x H ^ K which are uniformly continuous and bounded 
together with their first time derivative ft and its first and second space derivatives V/, V^/. 

If AT = R we do not write it in all the above spaces. 


2.2 C-derivatives 

We now introduce the C-directional derivatives following e.g. [25] or [36], Section 2. Here H, K, Z are 
Hilbert spaces. 


Definition 2.1 Let C : K ^ H be a bounded linear operator and let f : H ^ Z. 

• The C-directional derivative at a point x £ H in the direction k £ K is defined as: 

f{x -b sCk) - f{x) 


V‘"/(a;; k) = lim 


s e 


( 2 . 1 ) 


provided that the limit exists. 


We say that a eontinuous function f is C-Gateaux differentiable at a point x £ H if f admits 
the C-directional derivative in every direction k £ K and there exists a linear operator, called the 
C-Gdteaux differential, f{x) £ £(K,Z), such that f{x\k) = f{x)k for x £ H, k £ K. 
The function f is C-Gdteaux differentiable on H if it is C-Gdteaux differentiable at every point 
x£H. 
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• We say that f is C-Frechet differentiable at a point x € H if it is C-Gateaux differentiable and if 
the limit in 112.1\} is uniform for k in the unit ball of K. In this case we call f{x) the C-Frechet 
derivative (or simply the C-derivative) of f at x. We say that f is C-Frechet differentiable on H 
if it is C-Frechet differentiable at every point x € H. 

Note that, in doing the C-derivative, one considers only the directions in H selected by the image of 
C. When Z = R we have V‘^/(x) € K*. Usually we will identify K with its dual K* so V‘^/(x) will be 
treated as an element of K. 

If / : —?► R is Gateaux (Frechet) differentiable on H we have that, given any C as in the definition 

above, / is C-Gateaux (Frechet) differentiable on H and 

(V^/(a:),fc)^ = (V/(x),Cfc)^ 

i.e. the C-directional derivative is just the usual directional derivative at a point x £ H m. direction 
Ck G H. Anyway the C-derivative, as defined above, allows us to deal also with functions that are not 
Gateaux differentiable in every direction. 

For our purposes (to treat the case of pointwise delay) we need to extend the concept of C-derivative 
to the case when C : K ^ H is a closed linear and (possibly) unbounded operator with dense domain. 

Definition 2.2 Let f : H ^ Z. Let C : DiC) Q K ^ H be a closed linear (possibly) unbounded operator 
with dense domain. 

• The C-directional derivative f{x\k) at a point x € H in the direction k G D{C) is defined 
exactly as in a. 

• We say that f is C-Gdteaux differentiable at a point x € H if f admits the C-directional derivative 
in every direction k G DiC) and there exists a bounded linear operator, the C-Gdteaux derivative 

f{x) G K*, such that V‘^/(x; fc) = f{x)k for x G H and k G D{C). We say that f is 
C-Gdteaux differentiable on H if it is C-Gdteaux differentiable at every point x G H. 

• We say that f is C-Frechet differentiable at a point x G H if it is C-Gdteaux differentiable and if 
the limit in h2.1\) is uniform for k in the unit ball of K intersected with D{C). In this case we call 

f{x) the C-Frechet derivative (or simply the C-derivative) of f at x. We say that f is C-Frechet 
differentiable on H if it is C-Frechet differentiable at every point x G H. 


Remark 2.3 In this last case, even if f is Frechet differentiable at x G H, the C-derivative may not 
exist in such point. Indeed consider the following case. Let H be a Hilbert space and C : D{C) <G H ^ H 
be a closed linear operator on H with dense domain and with unbounded adjoint C* on H whose domain 
is D{C*). Let f : H ^ M. be Frechet differentiable at all x G H (e.g. f{x) = \x\^). By definition of 
C-directional derivative we have, for every x G H and h G D(C), that 

V^f{x-h) = {Vf{x),Ch)H. 

On the other hand, if the C-derivative of f exists at x G H then we should have f{x) G £(iJ;R) = H* 
(that we identify with H). Hence, if f was C-differentiable in all H this would imply that, for any x G H, 

\V^f{x- fc)| = I {V^f{x),k)^ I < c|fc|, Vfc G D{C). 

This would mean that V/(a:) G D{C*) for all x G H, which cannot be true in our example. 

Now we define suitable spaces of C-differentiable functions. 

Definition 2.4 Let I be an interval in R and let H, K, Z be suitable real Hilbert spaces. 

• We call Cl’'^{H, Z) the space of all functions in Cb{H,Z) which admit continuous and bounded 

C-Frechet derivative. Moreover we call {I x H,Z) the space of functions f G Cb{I x H,Z) 

such that for every t G I, f{t, •) G C^’^{H, Z). When Z = M. we omit it. 

• We call C^’^[H,Z) the space of all functions f in C({H,Z) which admit continuous and bounded 
directional second order derivative V^Vf; by C'°’^’^(/ x H,Z) we denote the space of functions 
f G Cb{I X H, K) such that for every t G I, f{t, ■) G C^’^{H, Z). When Z = ]& we omit it. 
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• For any a € (0,1) and T > 0 we denote by T] x H) the space of functions f € Cb{[0,T] x 

H,Z) n T] X H) such that the map (t,x) i—?► t°‘y^f{t,x) belongs to Cb{{0,T] x H,K*). 

The space {[0,T] x H) is a Banach space when endowed with the norm 

ll/llc2’i’'^(foTixm = sup \f{t,x)\+ sup ||V'^/(t,x)||^. . 

(t,x)elO,T]xH {t,x)G{0,T]xH 

When clear from the context we will write simply ||/||(^o,i,c. 

• For any a G (0, 1) and T > 0 we denote by T] x H) the space of functions f G (^{(([OjT] x 

-ff)nC'°’^’‘^((0, T] X iJ) such that for all t G (0,T], x & H the map {t^x) i—>■ f{t,x) is bounded 

and continuous as a map from (0,r] x FI with values in F{* x K*. The space T] x F[) 

turns out to be a Banach space if it is endowed with the norm 

\\f\\c°’^''^{[0,T]xH) 

= sup \f{t,x)\+ sup ||v/ (t,a;)||^. + sup ^“ ||a;) || 

(t.a:)e[0.r]xff {t,x)GlO,T]xH {t,x)GlO,T]xH 

3 Setting of the problem and main assumptions 

3.1 State equation 

In a complete probability space (II, P) we consider the following controlled stochastic differential equa¬ 
tion in R" with delay in the control: 

( dy(t) = aoy{t)dt + bQu(t)dt + bi{f)u(t + f)df + crdWt, t G [0, T] 

S 2/(0) = 2/0, (3-1) 

[ u{f) = uoif), f G [-d,0), 
where we assume the following. 

Hypothesis 3.1 

(i) W is a standard Brownian motion in R^', and {J-t)t>o is the augmented filtration generated by W; 
(a) ao G ^(R^iR"), cr is m£(R*^;R”); 

(Hi) the control strategy u belongs to U where 

U:={ze L^{n X [0,T],M™) : u{t) G U a.s.} 
where U is a closed subset o/R"; 

(iv) d> 0 (the maximum delay the control takes to affect the system); 

(v) bo G £(R™;R’"); 

(vi) bi is an mxn matrix of signed Radon measures on [—d, 0] (i.e. it is an element of the dual space of 
C([—d, 0], £(R™; M")) and the integral J^^bi(f)u{t + f)df is understood in the sense J^^bi{df)u(t + 
f) (bi is the density of the time taken by the control to affect the system). 

Notice that assumption (vi) on bi covers the very common case of pointwise delay but it is technically 
complicated to deal with: indeed it gives rise, as we are going to see in next subsection, to an unbounded 
control operator B. In some cases we will use the following more restrictive assumption on the term bi 
which leaves aside the pointwise delay case: 


6i G L2([-d,0],/:(R™;R”)). 


(3.2) 


Remark 3.2 Our results can be generalized to the case when the process y is infinite dimensional. More 
precisely, let y be the solution of the following controlled stochastic differential eguation in an infinite 
dimensional Hilbert space H, with delay in the control: 


dy{t) = Aoy{t)dt + Bou{t)dt + Bi(^}u(t + ^)d^ + adWt, t e [0, T] 

2 /( 0 ) = 2 / 0 , (3-3) 

u{0 = M0, ^&[-d,0). 


Here W is a cylindrical Wiener process in another Hilbert space S, and {iFt)t>o is the augmented filtration 
generated by W. Aq is the generator of a strongly continuous semigroup in H. The diffusion term a is 
in £(S; H) and is such that for every t > 0 the covariance operator 


e^^°aa*e^^«ds 


of the stochastic convolution 


is of trace class and, for some 7 € ( 0 , 1 ), 


/ 




s '^Tre'^^°aa*e^^°ds < + 00 . 


The control strategy u belongs to x [0,T],C/i), where Ui is another Hilbert space, and the space 

of admissible controls U is built in analogy with the finite dimensional case requiring control strategies 
to take values in a given closed subset U of Ui. On the control operators we assume Bq € C(Ui',H), 
Bi : [—d, 0] —>■ C{Ui,H)) such that Biu € L^([—d, 0],R) for all u £ 270 In this case, following again 
the problem can be reformulated as an abstract evolution equation in the Hilbert space TL that 
this time turns out to be H x L'^{[—d,0],H). All the results of this paper hold true in this case, under 
suitable minor changes that will be clarified along the way. I 


3.2 Infinite dimensional reformulation 

Now, using the approach of [44] (see [29] for the stochastic case), we reformulate equation (I3.1|l as an 
abstract stochastic differential equation in the Hilbert space TL = K" x L^([—d, 0], R"). To this end we 
introduce the operator A : 'D{A) C TL ^ TL as follows: for (?/o, 2/i) £ ^ 


A{yo,yi) = {aoyo + 2/i(0), -y'l), V{A) = {(yo,2/i) &'H-yi& lT^’^([-d,0],R”),yi(-d) = O} . (3.4) 
We denote by A* the adjoint operator of A: 

A*iyo,yi) = (ao2/o,2/(), = {(yo,2/i) e 7^ : 2/i e lH'’'([-d,0],R”),2/i(0) = yo} . (3.5) 

We denote by e*^ the Co-semigroup generated by A: tor y = {yo,yi) & TL, 


e 


tA 


yo 

yi 


e‘““2/o + /°d l[-t.o]e^*+®^““2/i(s)ds 

2 /i(- -t)l[_d+t.o](-)- 


(3.6) 


We will use, for iV £ N big enough, the resolvent operator (N — A) ^ which can 
giving 


(iV - A)-^ ( 

\ yi 


(iV-ao)-i 


2/0 + /°de^'*2/i(s)t^s 


■) 2 /i(s)ds. 


be computed explicitly 


(3.7) 


seems possible also in this case, to cover the case when Bi is a measure, on the line of Hypothesis [33(vi), but we 
do not do this here for simplicity. 
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Similarly, denoting by = (e‘^)* the Co-semigroup generated by A*, we have for z = (zq, zi) G 'H 


tA* 


ZO 

Zl 


e*“o Zq 


e(’^*)“ozol[_t_o](‘) + Zi{- + -*)(■)■ 

The infinite dimensional noise operator is defined as 

G:R^ ^ n, Gy = {ay, 0), y G R^. 

The control operator B, when (13.21) holds, is bounded and dehned as 

B:R"^->n, Bu= {bou,bi{-)u), uGR'^ 


(3.8) 


(3.9) 


(3.10) 


and its adjoint is 

B*:n*^R"^, B*{xo,xi) = b*xo+ [ bi{0*xi{0d^, {xo,xi) GH. (3.11) 

J-d 

In the general case B is possibly unbounded and defined as (here we denote by G*{\—d, 01, R") the dual 
space of C([-d,0],R”)) 


B : R™ ^R” X C*([-d,0],R"), 

n (3-12) 

{Bu)o = bou, (/,(Sn)i)c,c* =/l/(0&i(rfe)«, « e / G C([-d,0],R"). 

In this case the adjoint B* is (here we denote by G**{[—d, 0],R”) the dual space of G*{[—d, 0], R"), which 
contains G{[—d, 0],R")) 


(3.13) 


B* -.R^ X C**([-d,0],R”) ^ R™, 

B*{xo,xi) = b^xo + J^j^bi{dC)*xi{^)d^, (xq, a:i) G R” x C([-d, 0],R”). 

It will be useful to write the explicit expression of the first component of the operator as follows 

rO 


(e‘^S) :R™^R”, {e*^B) u = 3*^^°bou + I[_t.o]e(‘+’'>“‘’&i(dr)u, w G R™. (3.14) 

J-d 

Given any initial datum (yo, uq) G 'H and any admissible control u G U we call y{t] yg, uq, u) (or simply 
y{t) when clear from the context) the unique solution (which comes from standard results on SDE’s, see 
e.g. [22] Chapter 4, Sections 2 and 3) of (13.11) . 

Let us now define the process Y = (Fo,Li) G L^{fl x [0,T],H) as 


Yo{t) = y{t), 


Yiitm 



biidOuiC + t - 


where y is the solution of equation (ED), u is the control process in (13.11) and bi{dQ is understood as 
bi{C)dC when (13.21) holds. By Proposition 2 of [29], the process Y is the unique solution of the abstract 
evolution equation in 'H 


f dY(t) = AY(t)dt + Bu{t)dt + GdWt, t G [0, T] 
I Y{Q)=y= (yo,2/i), 


where yo = xq and yi(.^) = J^^bi{d()uo{C — ^). Note that we have yi G L^([—d,Oj;R" j3. Taking the 
integral (or mild) form of (I3.15|) we have 


Y{t) = e*^y+ [ e^*-^^^Bu{s)ds+ [ ed-^'i^GdWs, t G [0,T]. 

Jo Jo 


^This can be seen, e.g., by a simple application of Jensen inequality and Fubini theorem. 


(3.16) 
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3.3 Optimal Control problem 

The objective is to minimize, over all controls in U, the following finite horizon cost: 


J{t,x,u)=Ej^ [£o{s,y{s)) + £i{u{s))) ds+ E(j){x{T)). 


(3.17) 


where £o : [0, T] x R” —R and ^ : R" —R are continuous and bounded while : ?7 —R is measurable 
and bounded from below. Referring to the abstract formulation (13.151) the cost in (I3.17|) can be rewritten 
also as 


J{t,x;u) =E (^j\£ois,Yis)) + £,{uis))] ds + (T))^ , 

where £o : [0, T] x H R, t'l :[/—>■ R are defined by setting 

£o{t,x) := £o{t,XQ) Wx = {xo,xi) gTL 

£i ■■= £i 

(here we cut the bar only to keep the notation homogeneous) while (p : TL ^ E is defined as 

cj){x) := p{xo) Va; = (xo,xi) € H. 


(3.18) 

(3.19) 

(3.20) 

(3.21) 


Clearly, under the assumption above, £o and 4> are continuous and bounded while £i is measurable 
and bounded from below. The value function of the problem is 


V{t,x) := inf J{t,x]u). (3.22) 

u^U 

As done in Subsection o we define the Hamiltonian in a modified way (see dni)); indeed, for p £ "H, 
u € U, we define the current value Hamiltonian Hcv as 


Hcv{p;u) := {p,u)^m +£i{u) 
and the (minimum value) Hamiltonian by 

Hrmn{p) = inf Hcvip^u), (3.23) 

u^U 

The associated HJB equation with unknown v is then formally written as 

[ ^ GG*Y'^v{t,x) + (Ax,Y'v{t,x))^ +£o(t,x) + Hrmn{Y'^v{t,x)), t G [0,r], X G D{A), 

[ v{T,x) = (pix). 

(3.24) 

To get existence of mild solutions of (13.241) we will need the following assumption. 

Hypothesis 3.3 

(i) p £ CbiT-L) and it is given by h3.21\] for a suitable p G (^^(R"); 

(ii) £o G (^^([OjT] X T-L) and it is given by S3.19\) for a suitable £o G Ci,{{£t,T] x R"); 

(in) £i U ^ E is measurable and bounded from below; 

(iv) the Hamiltonian Hmin ■ > R is Lipschitz continuous so there exists L > 0 such that 

\Hmin{Pl) - Hmin{P2)\ < L\pi - P2\ ^ Pi, P 2 & 

(3.25) 

\Hrmn{p)\ < HI + \p\) VpGR™. 

To get more regular solutions (well defined second derivative V^V, which will be used to prove existence 
of optimal feedback controls) we will need the following further assumption. 

Hypothesis 3.4 
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(i) io is continuously differentiable in the variable x with bounded derivative. 

(ii) the Hamiltonian Hmin '■ —>• M is continuously differentiable and, for a given L > 0, we have, 

beyond Ii3.25\) . 

\yHrmnipi) - yHmin{P2)\ < L\Pl - P2\ Vpi,P2GR'"; (3.26) 

Remark 3.5 The assumption \3.25\) of Lipschitz continuity of Hmin is satisfied e.g. if the set U is 
compact. Indeed, for every pi, P 2 £ R™ 

\Hjnin{pi) - Hrmn{P2)\ < \ {Pl,u) - {p2,u) |, U&U 

and in the case of U compact the Lipschitz property immediately follows. The Lipschitz continuity of 
Hmin is satisfied also in the case when U is unbounded, if the current cost has linear growth at infinity. 

Moreover the assumption i3.26V of Lipschitz continuity of VHmin is verified e.g. if the function l\ 
is convex, differentiable with invertible derivative and with Lipschitz continuous since in this case 

{£[)-ffp)=VHrmn{p). ■ 


Remark 3.6 We list here, in order of increasing difficulty, some possible generalization of the above 
assumptions and of the consequent results. 


(i) All our results on the HJB equation and on the control problem could be extended without difficulties 
to the case when the boundedness assumption on f and Iq (and consequently on (f and io) can be 
replaced by a polynomial growth assumption: namely that, for some N € N, the functions 


X I—>• 




{t,x) 


1 + |a;|^ ’ 


(3.27) 


are bounded. The generalization of Theorem \^.1\ to this case can be achieved by straightforward 
changes in the proof, on the line of what is done, in a different context, in or in JSTH. 


(ii) Since our results on the HJB equation are based on smoothing properties (proved in Section^ which 
holds also for measurable functions, we could consider current cost and final cost only measurable 
instead of continuous. The proofs would be very similar but using different underlying spaces. 


(Hi) Using the approach of l!23jl it seems possible to relax the Lipschitz assumptions on the Hamiltonian 
function asking only local Lipschitz continuity of the Hamiltonian function, but paying the price of 
requiring differentiability of the data. 


In this paper we do not perform all such generalizations since we want to concentrate on the main point: 
the possibility of solving the HJB equation and the control problem without requiring the so-called 

structure condition. 


4 Partial smoothing for the Ornstein-Uhlenbeck semigroup 

This section is devoted to what we call the “partial” smoothing property of Ornstein-Uhlenbeck transition 
semigroup in a general Hilbert space. First, in Subsection l4.ll we give two general results on it (Theorem 
14.11 for the first C-derivative and Proposition 14.51 for the second derivative); then in Subsection 14.21 we 
prove two specific results for our problem (Propositions 14.91 and 14.111) . 


4.1 Partial smoothing in a general setting 

Let H, S be two real and separable Hilbert spaces and let us consider the Ornstein-Uhlenbeck process 
X^f) in H which solves the following SDE in H: 

r dX(t) = AX(t)dt + GdWt, t>0 , . 

1x(0) = x, 


where A is the generator of a strongly continuous semigroup in H, {Wt)t>o is a cylindrical Wiener process 
in S and G : 'E. ^ K. In mild form, the Ornstein-Uhlenbeck process X^ is given by 


X^{t) 


= e^^x+ [ e^*-^^^GdWs, 
Jo 


t > 0 . 


(4.2) 
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X is a Gaussian process, namely for every f > 0, the law of X{t) is Qt), the Gaussian measure 

with mean and covariance operator Qt, where 

Qt= [ e^^GG*e^^'ds. 

Jo 

The associated Ornstein-Uhlenbeck transition semigroup Rt, is defined by setting, for every / G Bb{H) 
and X € H, 

Rt[f]{x) = EfiX^t)) = [ fiz + e*^x)Af{0, Qt){dz). (4.3) 

Jk 

where by X^ we denote the Ornstein-Uhlenbeck process above with initial datum given hy x € H. 

It is well known (see e.g. m Section 9.4]), that Rt has the strong Feller property (i.e. it transforms 
bounded measurable functions in continuous ones) if and only if 

Ime*"^ C ImQ^^, (4-4) 


and that such property is equivalent to the so-called null-controllability of the linear control system 
identified by the couple of operators {A, G) (here z(-) is the state and a(-) is the control): 

z'{t) = Az{t) + Ga{t), z{Q) = X. 

(see again [U Appendix Bj). Under (14.41) Rt also transforms any bounded measurable function / into a 
Frechet differentiable one, the so-called “smoothing” property, and 

l|vi?t[/]lloo< ||r(t)||£(ff)||/||oo 

where r(t) := 

Here we extend this property in two directions: searching for C-derivatives and applying Rt to a 
specific class of bounded measurable functions, (see [35] for results in this direction in finite dimension). 

Now, let P : —>■ i? be a bounded linear operator; given any 0 : Im(P) —R measurable and bounded 

we define a function (j) G Bb{H), by setting 

(f>{x) = (j){Px) Vx G H. (4.5) 


We now prove that, under further assumptions on the operators A, G, C and P, the semigroup Rt maps 
functions 4>, defined as in (14.51) . into G-Frechet differentiable functions. 


Theorem 4.1 Let A be the generator of a strongly continuous semigroup in H and G : S —^ H. Let K 
he another real and separable Hilbert space and let G : D(C) Q K ^ H be a linear (possibly) unbounded 
operator with dense domain. Let cj) : Im(P) —>■ R. fee measurable and bounded and define : H ^ M. as in 
Fix t > 0 and assume that Pe^^C : K ^ H is well defined. 

Then, Rt[(f\ is G-Frechet differentiable if 


Im {Pe*^G) 

In this case we have, for every k G K, 

{X^{Rt[(l)]){x),k)K = [ fi{Pz + Pe^^x) 

JH 

Moreover for every k G K we have the estimate 

\{v^{Rt[4>]){x)k)^\<m^ 


C 


(4.6) 

'^Qf^^'^Pj^Ck, Qt){dz). 

(4.7) 


\k\K, 

C(K-H) 



Proof. Let k G D{C) C K. We compute the directional derivative in the direction Gk. 


lim — [i?t[())](a; -|- aGk) — Rt[cj)]{x)] 

a-i-O a 


= lim — 

Q :—>^0 Q . 


[ (t){Pz + Pe^^{x + aGk))M{0,Qt)idz)- [ {Pz + Pe^^x) M iO,Qt) (dz) 

JH JH 


lim — 
Q!^o a 


fH 


(j){Pz 


Pe^'^x) N {Pe*^aGk,Qt) [dz) — / (yPz-\-Pe*^x) M {Q,Qt) {dz) 

JH 
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By the Cameron-Martin theorem, see e.g. m, Theorem 1.3.6, the Gaussian measures M [Pe^^aCk,Qt) 
and JV (0, Qt) are equivalent if and only if Pe*^aCk G In such case, setting, for y G ImQ^^, 

- 1 , (4-8) 

we have, arguing exactly as in m, proof of Theorem 9.26, 

^ d(t, Pe*^aCk^ z) — I 


V‘^(i?t[(/>])(x)fc = lim — f (j) (Pz + Pe^^x) 
a^oaJH ^ 

= [ ^Pz + Pe*^x)(Q;^^^Pe*^Ck,Q;^^^z) M{Q,Qt){dz) 
Jh ' ' H 


M{Q,Qt){dz) 


which gives (lO) . Consequently 


{V^{Rt M)(x),fc)^|< 11^1 


' H 


Q-^l'^Pe^^CKQT'^z) N{Q,Qt){dz) ) = UU\\Qi"‘'Pe^''Ck\\a^K,Hy 


1-1/2. 


1/2 


This gives the claim. 


□ 


Remark 4.2 In \llf . Remark 9.29, it is showed that the analogous of condition is also a necessary 
condition for the Frechet differentiability of Rtlff] for any bounded Borel (f. In this case this is not obvious 
as we deal with a special class of (j) and so the counterexample provided in such Remark may not belong 
to this class. 


Remark 4.3 ITe consider two special cases of the previous Theorem \4.1\ that will be useful in next section. 

(i) Let K = H and C = I. In this case Theorem \4.1\ gives Frechet differentiability: for t > 0 Rt[0] is 
Frechet differentiable if 

Im G IniQ^^ (4.9) 

and we have, for every h G H, 


{ViRtmix),h)H = [ ^{Pz + Pe*^x)(Q;^^^Pe*^h,Qf^^h) J\fiO,Qt)idz). (4.10) 
J H ' ' ^ 


Moreover for every h G H we have the estimate 


{v{Rtm{^),h)H\<m 


Q-^fZp^tA 


C(H-H) 


\h\H, 


(ii) Let Kq and Ki be two real and separable Hilbert spaces and let K = Kq x Ki be the product space. 
Now, given any (j) G Bi,{Kq), we define, in the same way as in \3.21\) . a function (f> G Bi,{K), by 
setting 

(fik)=^{ko) yk = {ko,ki)GK. (4.11) 

Let P : K ^ Kg be the projection on the first component of K: for every k = (kg,ki) G K, 
Pk = kg. Theorem \4-. 1\ says that Rtlf}] is C-Frechet differentiable for every t > 0 if 

Im ((e‘^C')Q ,0) G ImQy^ Vt > 0 (4.12) 

and we have, for every k G K , 


{V^{Rt[mx).k)K= [ ^zg + ie^^x)o)(Q;^/^{{e^^Ck)^,0),Q;^^^z) Af{0,Qt)idz). 

J K ' ' ^ 


Moreover for every k G K we have the estimate 

I {V^{Rt [</>])(ai), I < Halloo {{e'^C)^,0) 


C(K-K) 


\k\ 


K, 


(4.13) 

(4.14) 
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Remark 4.4 In Theorem ED we prove the partial smoothing for functions (j) defined as in for 

functions </> bounded and measurable. The boundedness assumption on (p (and consequently on p) can be 
replaced by a polynomial growth assumption: namely that, for some N G N, 


X I—>■ 


1 + |a;|^ 


is bounded. The generalization of Theorem \4.1\ to this case can be achieved by straightforward changes in 
the proof, on the line of what is done in m or in 


4.1.1 Second derivatives 

We now prove that, if p is more regular, also Rt[(j)] and have more regularity. This fact, in the 

context of our model (see Subsection SSI, will be used in Section El to prove the Verification Theorem 
17.91 and the existence of optimal controls in feedback form in Theorem 17.131 


Proposition 4.5 Let A be the generator of a strongly eontinuous semigroup in H and G : S —> H. Let 
K be another real and separable Hilbert space and let C : D{C) C K ^ H be a linear (possibly) unbounded 
operator with dense domain. Let (j) : Im(P) —^ R 6e measurable and bounded and define (f> : H as in 
Fix t > 0 and assume that Pe*^C : K ^ H is well defined. Assume that holds true. If </> is 
such that (j) G Cl{H), then for every t > 0 the first order derivatives Rt [</>] and \7Rt {(fil exist and are 
bounded, with the second one given by 

{ViRt[cP]){x),h)jj = Rt[{Vfi,e*^h)^]{x), ^hGU. (4.15) 

Moreover the second order derivatives [fi], V^VRt [(p] exist, coincide, and we have 

(VV^(i?t [P]){x)k,h)^ = 

= (yp {Pz + Pe^^x) , Pe^^h)^ ((Qt)”'/" {Pe*^Ck) , iQt)-^/^z)^^{0, Qt){dz). (4.16) 

Moreover for every k G K, h G H we have the estimate 

I (VV^(i?t [4>]){x)k, h)^ I < \\VP\\oo\\Qf"'^Pe^^C\\c(K;H) \k\K \h\H. (4.17) 

Proof. We hrst prove (14.151) . Let (p : ImP —>■ R be such that, defining p as in (14.5L p G Cl{H). For any 
h G H we have, applying the dominated convergence theorem, 

f p[z + e^^{x + ah)) M {0,Qt) (dz) - f p [z + e*^x) M {0,Qt) (dz) 

H Jh 

= [ liiR — \p (z + e*^{x + ah)) — p (z + e^'^x)] Af {0,Qt) {dz) 
o'-' 


(Vi?t [p] {x),h)u = lim - 

CK—)-0 Ol 


= [ {yp{z + e*^x),e*^h)^Ar{0,Qt){dz) = Rt[{vp,e*^h)^] (x). 
Jh 


The boundedness of (V(i ?4 [p]){x),h) easily follows. We compute the second order derivatives starting 
from yy^ Rt[p]. Using (14.71) and the Dominated Convergence Theorem we get, ioi h G H ,k G K, 


h_m ^ [{y^{Rt [p]){x + ah)k)j^ - {y^{Rt [</>])(a;), fc)^] = 

= lim i y {p (Pz + Pe^^{x + ah)) - p {Pz + Pe^^x)) {{Qt)~^^‘^ (Pe^'^Ck) , {Qt)~^^'^z'^^Af{0, Qt){dz) 

= [ {yp{Pz + Pe^^x) ,Pe^^h)^(iQt)-^/^Pe*^Ck),iQt)-^/^z) jyiO,Qt){dz), 

Jh ' ' H 
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Similarly, using (14.151) and (14.5L we get, ioi h G H ,k G K, 


lim - [(V(i?t [cP]){x + aCk), h)^ - {V{Rt [</-])(a;), h)= 

a-s-O a 

= ^im i J ((V0 {Pz + Pe‘^(x + aCk)) , Pe*^h) ^ - (V^ {Pz + Pe*^x) , Pe*^h)^) AfiO, Qt){dz) 

= [ {V^{Pz + Pe*^x),Pe*^h} (iQt)-^/^Pe*^Ck),iQt)-^^^z) M{Q,Qt){dz). 

J H ' ' H 

The above immediately implies (I4.16|) and the estimate (14.171) . □ 


4.2 Partial smoothing in our model 

In the setting of Section [3] we assume that Hypothesis 13.11 holds true. We take 71 = K" x T^(—d, 0;R"), 
S = R^', (H, P) a complete probability space, W a standard Wiener process in S, A and G as in (13.41) 
and dSH). Then, for x G TL,we take the Ornstein-Uhlenbeck process X^{-) given by The associated 

Ornstein-Uhlenbeck transition semigroup Rt is defined as in (14.3p for all / G Bi,{R). 

The operator P of the previous subsection here is the projection Hq on the first component of the 
space R, similarly to Remark |4.3I (ii). Hence, given any (p G Hb(R"), we define, as in p.21l) a function 
(j) G Bb{R), by setting 

(j){x) = ^(Hqx) = P{xq) 'ix = {xq,xi) G R. (4-18) 

For such functions, the Ornstein-Uhlenbeck semigroup Rt is written as 


RtMx) = ¥.p{X%t)) = E^((W"(t))o) = [ K{z + e*^x)oW{0, Qt){dz). (4.19) 

Jn 

Concerning the covariance operator Qt we have the following. 

Lemma 4.6 Let A and G he defined respectively by i3.4\ ) and by iTOI) and let t > 0. Let be the 
selfadjoint operator in R" defined as 


Then for every {xo,xi) GR we have 


qO _ f ds. 

Jo 


Qt {xo,xi) = (Q°xo,0) 


(4.20) 


(4.21) 


and so 

Im Qt = ImQ° X {0} C R” x {0} 

Hence, for every G i3&(R") and for the corresponding :R ^ defined in we have 


RtW\{x) = f (l){zQ + {e*^x)Q)Af{0,Qt){dzo). 


(4.22) 


Proof. Let (a:o,xi) G R and t > 0. By direct computation we have 


Q( ^ ^ e^^GG*e^^‘ ^ ds 



from which the first claim (14.211) follows. The second claim (14.221) is immediate. 


□ 
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Remark 4.7 The statement of the above lemma holds true (substituting R" with the Hilbert space Kq 
introduced below) also in the following more general setting. Let % = Kq x Ki where Kq and Ki are both 

real separable Hilbert spaces. Let S be another separable Hilbert space (the noise space) and consider the 

Ornstein- Uhlenbeck process 

X{t) = e*^x+ [ e^*-‘‘'>^GdWs, t>0, (4.23) 

Jo 

where A generates a strongly continuous semigroup on H. and G € C((E.,'H). Assuming that 

• G = (cr, 0) for a G C{S,Kt)) so GG* = ^ q'^ q ^ ^ 

• for every ko G Kq, t > 0, 

e‘^(/co,0) = (e‘^«fco,0), (4.24) 

where Aq generates a strongly continuous semigroup in Kg; 


then the claim still hold. Indeed in such case we have, for t > 0, kg G Kg, ki G Ki, 

ikg,ki)j^ = 

where Aq is the adjoint o/^o@ So, for t>0. 


Q°tkg 





and 

Qt{kg,ki) = (Q°fco,0). 
This works, in particular, in the case described in Remark \3.‘A 


(4.25) 


We now analyze when Theorem 14.II can be applied in the cases G = I or C = B concentrating on the 
cases when the singularity at t = 0+ of \\Qf^^^I].ge*^G\\ is integrable, as this is needed to solve the HJB 
equation 

4.2.1 C = I 

By Theorem 14.II we have our partial smoothing (namely (I4.13P and (14.141) 1 for C = / if 

ImBoe*^ C ImQ^^ 


By Lemma [321 and (13.61) this implies 

Ime*““ C Im(Q°)i/^ (4.26) 

Since, clearly, is invertible and Im((3°)^/^ = ImQ°, then (14.261) is true if and only if the operator 
is invertible. On this we have the following result, taken from |46) [Theorem 1.2, p.l7] and [42) . 

Lemma 4.8 The operator Qt defined in is invertible for all t > 0 if and only if 

Im((T, aga ,..., ag'^tr) = R". 

This happens if and only if the linear control system identified by the couple (ag,a) is null controllable. 
In this case, for t —>■ 0+, 

where r is the Kalman exponent, i.e. the minimum r such that 

Im(cr, OoCT,. . . , OqCt) = R". 

Hence r = 0 if and only if a is onto. 

^Indeed once we know that e*'^(fco, 0)i = 0 then 114.2411 is equivalent to ask 114. 2511 . 
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We now pass to the smoothing property. 

Proposition 4.9 Let A and G be defined respectively by and E3). Let 4>: K” —>■ R &e measurable 

and bounded and define, as in —?► K, by setting (j){x) = 4>{xo) for every x = (a;oj 2 ;i) G %. 

Then, if Qt ** invertible, we have the following: 

(i) the function (t,x) i—>■ Rt[<p]{x) belongs to Ch((0,+oo) x TL). Moreover it is Lipschitz continuous in 
X uniformly in t & [toi^i] for all 0 < to < ti < +oo. 

(ii) Fix any t > 0. Rt[<p] is Frechet differentiable and we have, for every h € H, 

(V(i?t [(/)])(a;),h)« = [ fi{zo + {e*^x)o) {e*^h) Af{0,Q°t){dzo). 

(4.27) 


where [e*^x)^, (e^^h)^ are given by Eg). Moreover for every h £ TL we have the estimate 


{V{Rtm{x),h)^\<m\^ (e*^) 


£CH;R") 


l^lw- 


Hence for all T > 0 there exists Ct such that 

\{VRt[fi]{x),h)^\<CTt-^-^/^fi\Uh\n, t£[0,T], (4.28) 

where r is the Kalman exponent which is 0 if and only if a is onto. 


(Hi) Fix any t > 0. Rtlfi] is B-Frechet differentiable and we have, for every k £ R"*, 

(V®(i?f [0])(a;), = f 4>{zo-£ (e^^x) ) Ar(0, Q°)[dzo). 


Moreover, for every k £ 

|(V^(i?, [<(.])(^),fc)^„|<||^||, 

Hence for all T > 0 there exists Ct such that 
\{V^Rt [4>]ix),k}^„,\<C 
where r is the Kalman exponent which is 0 if and only if a is onto. 




(4.29) 


fc Rm. 

r:(ij™.R") 

(4.30) 


tG [o,r]. 

(4.31) 


Proof. Point (ii) immediately follows from the invertibility of Q^, the discussion just before Lemma H751 
and Theorem 14. II Point (i) follows from point (ii) and form the continuity of trajectories of the Ornstein 
Uhlenbeck process ED with A and G given by (|3.4I) and (13.911 . Point (iii) follows observing that the 
operator : R"* —>■ R", given in (|3.14l) is well defined and hence, thanks to the invertibility of Qt i 

Theorem 14.II can be applied. ■ 


4.2.2 C = B 

By Theorem 14.11 we have the partial smoothing (14.131 and (I4.14p for C = B if 

Im noe‘"^B C Im = Im Q° (4.32) 

Since, as proved e.g. in [46] (Lemma 1.1, p. 18 and formula (2.11) p. 210), 

Im((5t )^/^ = Im((T, aoCT,... Og” V), 
then, using (I3.14p . (I4.32p is verified if and only if 

Im ^e‘““6o + J l[-t,o]e!'^^'^'’°'°bi{dr'fj '£lTi\[a,ao(J,.. .ao~^(j). (4.33) 

We now provide conditions, possibly weaker than the invertibility of Qt, under which (I4.33|) is verified 
and the singularity at t = O'*' of \\Qf^^‘^IVoe*'^B\\ is integrable. We first recall the following result (see 
[46] . Proposition 2.1, p. 211). 
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Proposition 4.10 If Fi and F 2 are linear bounded operators acting between separable Hilbert spaces X, 
Z and Y, Z such that ||Fi*/|| = ||^ 2 */ll ^ Z*, then ImFi = ImF 2 and ||F]“^ 2 ;|| = ||-F' 2 ”^'^II 

all z G ImFi. 

Proposition 4.11 Assume that Hvvothesis \3.1\ holds. Assume moreover that, either 

Im(e‘“°6o) ^ Imcr, Vt > 0; 5i G L^(—d, 0, R”)), Im 6i(s) G Imtr, a.e. Vs G [—d, 0] (4.34) 


Im^e‘“‘’6o + J ^ Imcr, Vt > 0. 


(4.35) 


Then, for any hounded measurable 4> as in Rt {4>] is B-Frechet differentiable for every t > 0, and, 

for every h G R™, {V^{Rt [</>]) (a;), is given by ^4-29^ and satisfies the estimate {4-30^ ■ Moreover 

for all T > 0 there exists Ct such that 


I (V^(i?* mx), I < Crt-i/^ll^lloo |fck^. 

Proof. Consider the following linear deterministic controlled system in R: 

dX{f) = AX(t)dt + Gui{t)dt 
X(0) = Bh, 


(4.36) 


(4.37) 


where the state space is R, the control space is Ui = R^, the control strategy is ui G 
the initial point is Bh with h G R™. Define the linear operator 

£? :i2([0,t];t/i)^R”, ui{-) ^ [ e“‘>(‘-*Vui(s)ds. 

Then the first component of the state trajectory is 

X°{t) = Uoe*^Bk + C°ui (4.38) 

Hence X^ can be driven to 0 in time t if and only if 

Hoe^'^Hfe G Im£( 

In such case, by the definition of pseudoinverse (see Subsection EB, we have that the control which 
brings X^ to 0 in time t with minimal norm is {£^)~^IlQe^^Bk and the corresponding minimal square 
norm is 

£ if Bk) := min ki(s)|" ds : X (0) = Bk, X° (t) = o| = \\{C^t)-^Iloe^^Bk\\l,^^,.^^y (4.39) 

Since for all z G R” we have 

= I {Q°Z,z)^„ I = \\iC°,rz\\l^o,t;Uf 

then, by Proposition 14. 101 we get 


Im 




= im£l 


and 


\\i£°)-^U,e^^BkU2^o,t-uf = ||(g?)-'/"noe‘^Hfc||M.. 


Hence, by (14.391) . to estimate IKQ?) it is enough to estimate the minimal energy to steer 

X° to 0 in time t. When (14.341) holds we see, by simple computations, that the control 


Mi(s) = --cr ^e®““6ofc-(T ^bii-s)kl[^d^o]i-t), sG[0,t], 


(4.40) 


where tr ^ is the pseudoinverse of a, brings X° to 0 in time t. Hence, for a suitable C > 0 we get 

£{t,Bk)< J u?(s)ds < C ^-+ ||5 i||^2([_(; o];£(r™;R"))^ kiR™- 
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So, for a, possibly different constant C, we get ||(Qt) ||£(Rm.Rn) < Ct 2 |fc|Kn and the 

estimate is proved. If we assume (14.351) we can take as a control, on the line of [4^, Theorem 2.3-(iii), 

p.210, 

ui(s) = (^e‘““ 6 ofc + J , s G [0,t], (4.41) 

and use that the singularity of the second term as t —)► 0+ is still of order ^ since (14.351) holds (see e.g. 

[43], Theorem 1). Once this estimate is proved, the proof of the i?-Frechet differentiability is the same 
as the one of Proposition 14. bF ijib □ 

Remark 4.12 The above results can be generalized to the case, introduced in Remark \S.S\ above, when 
the first component of the space % is infinite dimensional. 

• For the case C = I the required partial smoothing holds if we ask, in place of the invertibility ofQ^, 
that, for every < > 0, 

Im(e*^)(j C Im(g°)i/2 (4.42) 

which would imply that the linear operator continuous from Ki into itself. 

• For the case C = B, the required partial smoothing holds if we ask that, for every t > 0, 

Im (e*'^R)p C Im(Q°)i/2 ( 4 . 43 ) 

which would imply that the operator (e*'^R)p is continuous from U to Ki. 

Clearly, in this generalized setting the estimates li4-28\ ) and Ii4.31\ ) do not hold any more and they depend 
on the specific operators A, B, a. I 


5 Smoothing properties of the convolution 

By Proposition 14.111 we know that if (I4.34|) or (14.351) hold and f) is as in (14.181) with f measurable and 
bounded then [i 5 !'])(x) exists and its norm blows up like t~^/^ at 0+. Moreover if ^ G (^^(IR"), then 

G Ch([0,T] X 'H), see e.g. HO] Proposition 6.5.1 (or the discussion at the end of HD). 

We now prove that, given T > 0, for any element / of a suitable family of functions in (^^([O, T]x'H), a 
similar smoothing property for the convolution integral /p Rt-s[f{s, •)](a;)ds holds. This will be a crucial 
step to prove the existence and uniqueness of the solution of our HJB equation in next section. 

For given a G (0,1) we define now a space designed for our purposes. 

Definition 5.1 Let T > 0, a G (0, 1). A function g G (^^([OjT] x %) belongs to ^ if there exists a 
function f G C°’^([0,T] x R”) such that 

9{t, x) = f {t, (e*^x)o) , V(t, x) G [0, T] x H. 

If g G a-! ^ £ (O’ function g{t, ■) is both Frechet differentiable and R-Frechet differentiable. 

Moreover, for {t,x) G [0, T] x "H, li G "H, fc G M™, 

{Vg{t, x),h)^ = (V/ (t, (e*^x)o) , (e*^/i)o)ju„ , and {V^gft, x),k)^^ = (V/ {t, (e‘^x)o) , (e‘"^Rfc)o)jj„ . 
This in particular imply that, for all k G 

(V^g(t,a:), Pj.jjnj , (5.1) 

which also means B*Vg = V^g. For later notational use we call / G Cb((0,T] x R";R.™) the function 
defined by 

(/>,2/),fc)^„ =t“(V/(t,2/),(e‘^Rfc)o)^„, (t,y)G(0,r]xR", fc G M™, 

which is such that 

V^g(t, x)= f (t, {e*^x)o) . 

We also notice that if g G then in order to have g R-Frechet differentiable it suffices to require 

(e*"^R)o bounded and continuous. 

When (I4.34P or (|4.35l) hold we know, by Proposition H tD that the function g(t,x) = Rt[(j)\{x) for (f> 
given by (I4.18|) with (f bounded and continuous, belongs to Sy ^^^ 2 - 
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Lemma 5.2 The set is a closed subspace of {[0,T] x TL). 

Proof. It is clear that is a vector subspace of T] x 'H). We prove now that it is closed. 

Take any sequence —?► g in T] x 'H). Then to every gn we associate the corresponding /„ and 

/„. The sequence {/„} is a Cauchy sequence in Cb{[0,T] x R"). Indeed for any e > 0 take {te,ye) such 
that 

sup \fnit,y) - fniit,y)\ < e+ \fn{te,ye) “ fm{te,ye)\ 

(t.i/)e[0,T]xR" 

Then choose &TL such that y^ = (this can always be done choosing e.g. 0)). 

Hence we get 


sup \fn{t,y) - fm{t,y)\ < f-+\9n{te,xf) - gm{te,xf)\<e+ SUp (t, x) - (t, x) |. 

(t,y)G[0,T]xR” (t,a:)e[0.T]xW 

Since {g„} is Cauchy, then {/„} is Cauchy, too. So there exists a function / G C'b([0,r] x R”) such that 
/n —>■ / in C'b([0, T] X R"). This implies that g(t, x) = f(t, (e‘"^x)o) on [0, Tjx'H. With the same argument 
we get that there exists a function / G C'b((0,T] x R";R™) such that /n —>■ / in C't,((0,T] x R";R"*). 
This implies that t“V^g(t,x) = /(t, (e‘"^x)o) on (0,r] x "H. □ 

Next, in analogy to what we have done defining Ey introduce a subspace of functions 

g G C'°’^’'®([0, T] X TL) that depends in a special way on the variable x G "H. 

Definition 5.3 A function g € Cb{[0,T] xTl) belongs toYf^^ if there exists a function f G C^’^{[0,T] x 
R") such that for all (t,x) G [0,r] x TL, 

g{t,x) = f {t, (e*^x)o) . 

If g G then for any t G (0,r] the function g{t, •) is Frechet differentiable and 

(Vg(t, x), h)^ = (V/ {t, (e‘^x)o) , , for (t, x) G [0, TjxTi, heH. 

Moreover also Vg{t, ■) is H-Frechet differentiable and 

(V^ (Vg(t, x)h)),k}^^ = (VV (t, (e^^x)o) (e‘^h)o, (e^'^Bk)o)^^ , for (t, x) e 10,T] x ?{, h e ?{, k e R'" 

We also notice that, since the function / is twice continuously Frechet differentiable the second order 
derivatives S/^S/g and both exist and coincide: 

(V^ {Vg{t, x), h)^ , k)^^ = (V {V^g{t, x),k)^^ , h)^ . 

Again for later notational use we call fi G C'6([0,r] x R";R™) the function defined by 

{fi{t,y),h)^^={Vf{t,y),{e*^Bh)o)^^, (t,y) G [0,T] x R", heR^, 

which is such that 

V^g{t,x) = fi {t, (e*^x)o) . 

Similarly we call f G Cb ((0,T] x R";£('H,R™)) the function defined by 
{{fit, y), h)^ , k)^^ = (V^/ (t, y) {e*^h)o, {t, y) G [0, T] x R", hG'H,kG R^, 

which is such that 

t“V^Vg(t, x) = VV^ 5 (t, x) = / (t, (e*^x)o) . 

We now pass to the announced smoothing result. 


Lemma 5.4 Let j4-d4 ) or i4-‘d5\ l hold true, 
satisfiying Hypothesis I13.3\) . estimates US. 25\} . 


Let T > 0 and let : R™ 
Then 


—^ R 1)6 a continuous function 
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i) for every g G the function g : [0, T] x 'H —?► R belongs to where 

g{t,x)=f Rt-s['ipi^^g{s,-))]{x)ds. (5.2) 

^0 

Hence, in particular, g{t, •) is B-Frechet differentiable for every t € (0,T] and, for all x G H, 

|v''(5(i,-))(a;)|(K™). + (5.3) 

If a is onto, then g(t, ■) is Frechet differentiable for every t G (0,T] and, for all h €11, x € H, 

|V(p(i,-))(a:)|^. <c(t^^'^ + \\g\\co.f • (5.4) 

ii) Assume moreover that ip G C'^(R’"). For every g G function g defined in i5.‘d\} belongs to 

^Ti/ 2 ' Hence, in particular, the second order derivatives g{t, ■) andV^Vg{t,-) exist, coincide 
and for every t G (0, T] and, for all x € H, 

|V^V(5(t,-))(^)|«.^(R^). < C\\g\\co...B (5.5) 

If (7 is onto, then g(t,-) is twice Frechet differentiable and for every t G (0,T], for all h £ % and 

X €11, 

|V^(p(i,-))(a:)|„.^„. <C'||g||c,o2 (5.6) 

1/2 

Proof. We start by proving that (15.21) is i3-Frechet differentiable and we exhibit its B-Frechet derivative. 
Recalling (14.31) we have 

f Rt-s[ip{V^{g{s,-)))]{x)ds= f j ip{y^{g{s,-))(^z + e^*~’''^^x'fjJ\f{Q,Qt-s){dz) 

t/ 0 '10 'I n 

By the definition of we see that 

s^/^V^ 5 (s, z -|- e^*~’^^^x) = f (s, (e'’"^z)o + {e*^x)o) Vf > s > 0, Vx, z € H. (5.7) 

Hence the function / associated to g is 

fit,y) = I^ / i^^i^"^^)o + y)^AfiO,Qt-s)idz) 

with, by our assumptions on ip, 


I|/||oo<c'£(i + s-i/ 2 ||/|u) ds 

To compute the B-directional derivative we look at the limit 

lim - [ Rt-s [ip (V^(g(s, •)))] (a; -f aBk)ds - [ Rts [ip (V^(g(s, •)))] {x)ds 

“^0 a Uo Jo 

From what is given above we get 

f Rt-s [ip (V^( 5 (s, •)))] (x + aBk)ds 
^0 

= yy ip{s~^/'^f {s,{e‘^^z)[) + {e^^{x + aBh))o)^N{i),Qt-s){dz)ds. 
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Since the last integrand only depends on {e^^z)Q + {e*^{x + aBk))o, we have, arguing exactly as in the 
proof of Theorem 14.11 

[ Rt-s [i> {v^{g{s, •)))] (x + aBk)ds = 

Jo 

= [ [ (e®^ 2 )o + (e*^x)o)) A/'(((e‘^aBfc)o,0) ,Qt-s) (dz)ds = 

'J 0 'J l-L 

= I J V’(e®^2)o + (e*^x)o)) - S, aBfe,z)A/'(0,(5t-s) (d-2:)ds, 


where 


d{ti,t 2 ,y,z) = - ^ - [z) 


Hence 


lim — 
a->0 a 


dj\f (0,QtJ 

= exp ((e‘^^y)o .0) " \ \Qi^'^ ((e*^^2/)o > O) 


f Rt-s [t/'(V^(5(s,-)))] {x + aBk)ds- f i?t_s [V'(V® (g(s, •)))] {x)ds 
Jo Jo 


(5.8) 


d(t, t — s, aBk, z) — 1 


A/'(0, Qt-s) {dz)ds 


= hm (s-'/V(s, (e*^z)o + (e*^x)o)) 

= ^ / '0 (e'’^2)o + (e*^a;)o)) {{J"^Bk)^,0) ,Q~y^^z'^^Jd' {0,Qt-s) {dz)ds. 

Since the above limit is uniform for k in the unit sphere, then we get the required H-Frechet differentia¬ 
bility and 


^(f Rt-s [tpy^igis,-yds) {x),k\ = (5.9) 

\J 0 / / R™ 

^ lo I ^ {iQt-s)~^^‘^ {J^Bk)^,{Q°_^)-^/'^zo'j^^Jd'{ 0 ,Qt-s){dz)ds. 

Finally we prove the estimate (15.3F Using the above representation and the Holder inequality we have 


^ (i) y^iais, •)))] ds^ (x), k^ 

(l + |s-i/V(s,(e^^^)o + (e‘^x)o)|) |((Q?_J-'/' {e^^Bk)^,{Q^,_,)-^/^zo)^^\M{0,Qt-s){dz)ds 

< cj ^ (i + s-i/2||5||po,i,b) ||(QO_J-i/2(e‘^Hfc)o 


^(RmiR") 


< 


C J ||g||pO,l,B^ (t - s) ^/^|/c|Rm ds < c + ||g||pO,l,B^ |h|Rm. 


Observe that in the last step we have used the estimate 


iQts)-^^^y^Bh)o 




C(R”*;R'*) 

which follows from the proof of Proposition 14.91 for Proposition 14.IIP Moreover we have also used that 

[\t - s)-i/2s-i/2^5 = f\i _ x)-^l^x-^l^dx = P (1/2,1/2), 

JO Jo 
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(5.10) 
























where by /3(-, •) we mean the Euler beta function. 

The Frechet differentiability and the estimate (15.41) is proved exactly in the same way using the fact 
that a is onto and Proposition 14.91 

Now we consider the case of g G We start by proving that (j5.2p is Frechet differentiable and, 

in order to compute the Frechet derivative, we use (I4.15|) (which is true for every (f> G Cl{H), see its 
proof) looking at the limit, ior h G TL, 


{Vg{t,x),h)^ = lim 


Q :—>-0 OL 


= / Rt- 


= Rf- 


Rt-s [tp •)))] {x + ah)ds - [ Rts [ip {^^{g{s, •)))] {x)ds 

Jo 

W {tp {V^{g{s,-)))) {x)ds 

' 'M.- 

Vp, (V^(g(s, •))) , VV® (5(s, 1 {x)ds 

^ J (wpj (V^{g{s, z + , VV® (g(s, ^ AT (0, Qts) {dz)ds 


(5.11) 


VV'(/i (s, (e'’^z)o + (e‘^a;)o)) ,s (s, (e®'^z)o + (e*'^a;)o) ] Af (O^Qtsidz)) ds 

Now, from calculations similar to the ones performed in the first part we arrive at 


(V® (V5(t,x),/i)„,fc)jj„ =y J (Vtp {fi {s,{e‘'^z)o +{e*^x)o)) ,s {s,{e''^z)o + {e*^x)o) 




ds. 


(5.12) 


Since is bounded (as it satisfies (13.251) ') then (15.51) easily follows by the definition of / and from (j5.10p . 
Second order differentiability and estimate (15.6L when a is onto, follow in the same way. □ 


6 Regular solutions of the HJB equation 

We show first, in Subsection 16.11 that the HJB equation (11.51) admits a unique mild solution v which is 
B-Frechet differentiable. Then (Subsection 16.21) we prove a further regularity result whose proof is more 
complicated than the previous one and that will be useful to solve our control problem in Section [7.31 


6.1 Existence and uniqueness of mild solutions 

We start showing how to rewrite (j3.24|) in its integral (or “mild”) form as anticipated in the introduction, 
formula (16.31) . Denoting by £ the generator of the Ornstein-Uhlenbeck semigroup i?t, we know that, for 
all / G Cl{'H) such that V/ G D[A*) (see e.g. [9] Section 5 or also [12] Theorem 2.7): 

mx) = \Tr GG* V^fix) + {x, A*yf{x)). (6.1) 

The HJB equation (11.51) can then be formally rewritten as 


dv(t, x) 


= C[v{t,-)]{x) +io{t,x) + Hmin{A7^v{t,x)), t G [0,T], xGAL, 


dt 

v{T,x) = (p{xo). 

By applying formally the variation of constants formula we then have 

v{t,x) = RT-t[(p]{x) + Rs-t[Hjnin{A7^v{s,-)) + io{s,-)] {x) ds, tG[0,r], xGH, 
We use this formula to give the notion of mild solution for the HJB equation (16.21) . 


( 6 . 2 ) 


(6.3) 
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Definition 6.1 We say that a function z; : [0, T] x ^ M is a mild solution of the HJB equation i6.2\} 
if the following are satisfied: 

1. {[Q,T]xn); 

2. equality i6.S\) holds on [0,T] x %. 

Remark 6.2 Since {[0,T] x B) C Cb{[0,T] x T-L) (see Definition I2.4P the above Definition \6.1\ 

requires, among other properties, that a mild solution is continuous and hounded up to T. This constrains 
the assumptions on the data, e.g. it implies that the final datum <f) must be continuous and bounded. 
As recalled in Remark \4.4[ (i) o-nd (ii) we may change this requirement in the above definition asking 
only polynomial growth in x and/or measurability of /. Most of our main results will remain true with 
straightforward modifications. 


Since the transition semigroup Rt is not even strongly Feller we cannot study the existence and 
uniqueness of a mild solution of equation (lOD as it is done e.g. in m- We then use the partial 
smoothing property studied in Sections U and [5] Due to Lemma l5.4l the right space where to seek a mild 
solution seems to be indeed our existence and uniqueness result will be proved by a fixed point 

argument in such space. 


Theorem 6.3 Let Huvotheses \3.1\ and \3.S\ hold and let Ii4.34\l or ^.35^ hold. Then the HJB equation 
admits a mild solution v according to Definition \6. 1\ Moreover v is unique among the functions w 
such that w(T —■,■)€ T^t,\/i and it satisfies, for suitable Ct > 0, the estimate 

||^;(T-.,.)|Ico,cb <Ct(||<^||oo + ||4||oo). (6.4) 

Finally if the initial datum/ is also continuously B-Frechet (or Frechet) differentiable, thenv G T] x 

H.) and, for suitable Ct > 0, 

Ikllco.i.B < Ct {UWoo + ||V^</)||oo + ||4||oo) (6.5) 

(substituting with V/ if / is Frechet differentiable). 


Proof. We use a fixed point argument in To this aim, first we rewrite (16.31) in a forward way. 

Namely if v satisfies (16.31) then, setting w(t,x) := v{T — t,x) for any {t,x) G [0,T] x %, we get that w 
satisfies 


w{t,x) = Rt[(t>]{x) + [ Rt-s[H,riin(^^'w{s,-)) + £o{s,-)]{x) ds, tG[0,T],xGH, 
Jo 


( 6 . 6 ) 


which is the mild form of the forward HJB equation 
dw(t, X 


dt 

w(0, x) = /(x). 


= £lw(t, ^((a:) + £o(t, x) + i?mm(V"w(t, x)), t G [0,T], x G H, 


(6.7) 


Define the map C on ^^^2 by setting, for g G E/ 

C{g){t,x) := Rt[(j)]{x)-\- [ io{s,-)]{x) ds, tG [0,T], (6.8) 

By Proposition 14.Ill and Lemma [5^ til we deduce that C is well defined in E^ ^^2 takes its values in 
^Ti/ 2 - Since in Lemma [5.21 we have proved that Ei)., j ^^2 is a closed subspace of Cy]f^{[0,T] x TL), once 
we have proved that C is a contraction, by the Contraction Mapping Principle there exists a unique (in 
E^ 1 / 2 ) fixed point of the map C, which gives a mild solution of (16.21) . 
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Let 51,52 e S;^,i/2- We evaluate ||C(5i) - C(52 )||e,j„i/ 2 = l|C(5i) - C(52)|lc'>'i'^- First of all, arguing as in 
the proof of Lemma lOl we have, for every (t,x) € [0,T] x H, 


\C{9i){'t,x) -C(52)(t,x)| = 


[ Rt-s [Hrmn (V®5i(s, •)) " (V®52(s, •))] {x)ds 

Jo 


< 


[ s sup |s^/^V^(5i-52)(s,5)|ds < 2Lt^/^||5i-5211(^0.1,-B. 

Jo yen 1/2 


Similarly, arguing exactly as in the proof of (15.3p . we get 

f Rt-s [Hrmn (V'®5l(s, •)) - Hmin (V'® 52 (s, •))] {x)ds 

Jo 

< t^/^L||5i - 52|lc°’i>ii / (^ “ (1/2,1/2) ||5i - 5211^.0 

1/2 Jo ^1, 




|, 1 ,B . 
1/2 


Hence, if T is sufficiently small, we get 


\\C{gi) - C(52)|lci>-I>ii ^ C llffl “ 52 |Ic°’1’11 


" 1/2 


" 1/2 


(6.9) 


with C < 1. So the map C is a contraction in Ei/ and, if we denote by w its unique fixed point, then 
V := w{T — •, •) turns out to be a mild solution of the HJB equation (16.21) . according to Dehnition 16.31 
Since the constant L is independent of t, the case of generic T > 0 follows by dividing the interval 
[0,r] into a finite number of subintervals of length 6 sufficiently small, or equivalently, as done in |36] . 
by taking an equivalent norm with an adequate exponential weight, such as 




sup |e''*/(i,a:)| + sup ||V®/(t,a;)||, , 

(t,a:)G[0,T]xW (t,x)e(0,T] X W ' ' 


The estimate (16.41) follows from Proposition 14.91 and Lemma [5^ 

Finally the proof of the last statement follows observing that, if (p is continuously B-Frechet (or 
Frechet) differentiable, then Rt[(p] is continuously B-Frechet differentiable with V^Rt[(j)] bounded in 
[0,r] X TL, see lemma H751 formula (14.151) . This allows to perform the fixed point, exactly as done in the 
hrst part of the proof, in {[0,T] x TL) and to prove estimate (j6.5l) . □ 


Corollary 6.4 Let Hwotheses I,*?.41 and \S.,'L\ hold and let a he onto. Then the mild solution of equation 
found in the previous theorem is also Frechet differentiable, and the following estimate holds true 

HT--,-)llcV <C'r(||</||oo + ||4||oo) (6.10) 

1/2 


for a suitable Ct > 0. 


Proof. Let v be the mild solution of equation (16.21) . and Vt € [0, T], a: € 'H define w{t, x) := v{T — t,x), 
so that w satisfies (EH), so that by applying the last statement of Lemma 15.41 it is immediate to see that 
w G xTL). By differentiating (16.6|) we get 


Vw(t,x) =VRt[(l)]{x) + tQ{s,‘)\{x) ds, tG[0,T], xGH, 

Jo 

By Lemma oi the above recalled variation of Lemma 15.41 and estimate (15.4|) , we get that 


|Vw(t,a;)| < Ct ^/^||(/>||oo + (^1 + HicII^^o.i.b + IKolU) , t G [0,T], x gTL, 


which gives the claim using the estimate for ||r(;||^o,i,B given in (16.41) . 

'" 1/2 


□ 
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6.2 Second derivative of mild solutions 


The further regularity result we are going to prove will be needed in Section [7.31 Theorem l7.13l A similar 
result can be found in m, Section 4.2. Here we use the same line of proof but we need to argue in a 
different way to get the apriori estimates. 

Theorem 6.5 Let Hvvotheses \S. 1[ r.9.,91 and \3.Ji\ hold. Let also Ii4-34V or Ii4-35\ l hold. Let v be the mild 
solution of the HJB equation f6‘.3l) as from Theorem \6.3[ Then we have the following. 

(i) Lf (j) is continuously differentiable then we have v G 2 / 2 ' hence the second order derivatives 


and WV^v exist and are equal. Moreover there exists a constant C > 0 such that 

|V?;(t,a;)| <C'(||V0||oo + ||V4||oo), (6.11) 

|V^V«(t,x)| = \VV^v{t,x)\<C ({T -t)-^/^V4>\\^ + {T . (6.12) 

Finally, if a is onto, then also exists and is continuous and, for suitable C > 0, 

I V\(t, x)| < C ((T - t)-i/2|| V0IU + {T- V4||oo) . (6.13) 

(ii) Iff is only continuous then the function {t,x) 1 —>■ {T — tY^^v{t,x) belongs to .^ 1 .^. Moreover there 
exists a constant C > 0 such that 

|Vu(t, x)| < C ((T - ty^^^fWoo + IIV4||oo) , (6.14) 

|V^Vu(t,x)| = |VV^u(t,a:)| <c((T-t)-i||0|U + (T-t)-i/2||V4||oo) • (6.15) 

Finally, if a is onto, then also V^v exists and is continuous in [0, T) x H and, for suitable C > 0, 
|V\(t,x)| < C ((T- t)-i||V^||oo + {T- t)-i/2||V4||oo) . (6.16) 


Proof. We start proving (i) by applying the Contraction Mapping Theorem in a closed ball Ht(0, R) (i? 
to be chosen later) of the space By Proposition 14.51 and Lemma [5.41 we deduce that the map C 

defined in (16.81) brings 1 / 2 - Moreover, for every g G we get, first using (I3.25L 


\C{g){t,x)\ < \Rt[(l)\ix)\ 


Rt-s [Hmin (V^g(s, •)) +4(s,-)] {x)ds 


< ll<('||oo +tL (1 + ||g||pO,l,B^ + t||fo||c 


second by (14.151) . (I5.11L (13.251) (calling M := sup[Q j.] ||e' 


tA\ 


|VC(5)(t,x)| < \^Rtmx)\ 


V [ Rt-s [Hrmn {V^9{s, ■)) + 4(s, •)] {x)ds 
Jo 


< 

< M 


M\\Vf\y +M [ [\\VHrmn (V^g(s, •)) VV^5(s, OL + II V4(s, Olloo] ds 
Jo 


\\37f\\oo+t\\Vio\\oo + Ley\gl 


"1/2 J 


(6.17) 


third by (14.171) (with (I4.34|) or (14.351) 1. and (15.51) 

d/2|yByc(g)(t,x)| < I[().](x) I + 


V^V [ Rt-s (V^g(s,-)) +4(s,-)] {x)ds 

Jo 


< C||V</.||oo+C'd/2 [ (t-s)-i/2||v4(s,.)||oods + Cd/2||g||po.2,B 

Jo I/" 


< C 


||v<^||oo + 2t||V4||oo + d/2|i5|| o,,.B 


1/2 J 


(6.18) 

(6.19) 
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with the constant C (that may change from line to line) given by the quoted estimates. Hence, for 
g G Bt{0,R), we get, for given Ci > 0, 


\\C{g)\l 


where we define 


■" 1/2 


<Ci 


<Ci 


UWcl +T + T\Mcl] + TL\\9\\^oa,B + {ML + C)T^^^\\g\\, 
\\cj,\\ci+T + T\\io\\cl] +P{T)R 


p{T) ■= TL + {ML + C)r^/^ 


" 1/2 


( 6 . 20 ) 

( 6 . 21 ) 


Now take 31,52 G Arguing as in the above estimates we have, for every it.x) G [0,T] x "H, 


\C{gi){t,x) -C{g 2 ){t,x)\ = 


[ Rt-s [Hmin (V^5i(s,-)) - Hmin (V^52(s, •))] {x)ds 
Jo 


< tA||3l -52||(;.0,1,B 


|VC(5i)(t,x) - VC(52)(t,x)| = V [ Rt-s [Hmin {^^gi{s, ■))-Hmin {^^g 2 {s,-))]{x)ds 

Jo 

< M f \\VHmin (V^3i(s, •)) VV^3i(s, •) - VHmin (V^52(s, •)) VV^52(s, Olloo 
Jo 


\\gi - 92 \\c°J’’^\\9i\\c°'^'’^ + II 51 - 52110“’,=’® 


" 1/2 


< 2MLt^/‘^ 
and, using (15.121) . 

ti/2|V^VC(5i)(f,x)-V^VC(52)(t,x)| V^V 


"1/2 J 


f Rt-s [Hmin (V^5i(s, •)) - Hmin (V^52(s, •))] {x)ds 
JO 


< t^/'^MLfi (1/2,1/2) II51 - 52||(^0,i,B||5i||p0,2,B + II51 - 52||o“’=’- 
L ^0 ^1/2 ^1/2 

Hence, for 51,52 G Bt{0, R), we have, recalling (16.211) and the way C is found in (16.191) . 

l|C(5i) - C(52 )|Ic“;=’®([o.t]x«) <l(t + MT^!^ (2 + /3 (1/2,1/2)) (1 + i?)) II51 - 52||po,2,r 

< p{T){^ + R) II 51 - 52|Ic“’=’® ^ 


"1/2 


( 6 . 22 ) 


ll'/’llci + T" + T’lKollci can find Tq sufficiently small 


6.221) . C is a contraction in Btq (0, R). Let then w be the 


Now, by (16.201) and (I6.22p . choosing any R> Ci 

so that p{Tq) < 1/2 and so, thanks to (16.201) and 

unique fixed point of C in Btq {0,R)' it must coincide with v{T — ■, ■) ior t G [0, Tq]. This procedure can be 

iterated arriving to cover the whole interval [ 0 ,r] if we give an apriori estimate for the norm ||ii;||^o,2,b . 

^ 1/2 

By the last statement of Theorem 16.31 we already have an apriori estimate for ||w||oo + || V^u;||oo- To get 
the estimate for Vw and we use the ()6.17p and ()6.19p where we put w in place of Cg and 5 . From 

the first line of (I6.19P and 15.121 we get 

||V^Vu;(t, OIU < C [||V<^||oo + 2T||V4||oo] + L f {t - s)-^/^\\V^Vw{s, OlUds 

which, thanks to the Gronwall Lemma (see m, Subsection 1.2.1, p. 6 ) give the apriori estimate for 
V^Vw. Then from the second line of (|6.17l) we get 

II Vu;(t, Olloo < M [||V/.||oo + T|| V4||oo] + ML [ || V^Vu;(s, ■)\\^ds 


which gives the apriori estimate for Vw using the previous one for V^Vw. Estimate (16.131) follows by 
repeating the same arguments above but replacing V^V with V^. 
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We now prove (ii). Let v be the mild solution of (16.21) and, for all e g]0,T[, x £ H, call (j)^{x) = 
v{T — £, x). Then v is the unique mild solution, on [0, T — e] x 7^, of the equation (for t £ [0,T], x £ H) 

rT-e 

v{t, x) = RT-e-t(l>‘'{x) + J Rs-t [Hrmn{^^v{s, •)) + 4(s, •)] {x)ds (6.23) 

This fact can be easily seen by applying the semigroup property of Rt (see e.g. m Lemma 4.10 for a 
completely similar result). 

Now, by Theorem 16.31 (j)^ is continuously differentiable, so we can apply part (i) of this theorem to 
(|6.23l) getting the required regularity. Estimates (I6.14|) - (I6.15|) - (I6.16I) follows using estimates (16.111) - 
(|6.12|) - (|6.13l) with (j)^ in place of (j) and then using the arbitrariness of e and applying (16.41) to estimate 
(()® in term of (f). □ 


7 Verification Theorem and Optimal Feedbacks 

The aim of this section is to provide a verification theorem and the existence of optimal feedback controls 
for our problem. This in particular will imply that the mild solution v of the HJB equation (16.21) built in 
Theorem 16.31 is equal to the value function V of our optimal control problem. 

The main tool needed to get the wanted results is an identity (often called “fundamental identitj /’, 
see equation (17.81) 1 satisfied by the solutions of the HJB equation. When the solution is smooth enough 
(e.g. it belongs to UCl’’^ (PT] x H)) such identity is easily proved using the Ito’s Formula. Since in 
our case the value function does not possess this regularity, we proceed by approximation. Due to the 
features of our problem (lack of smoothing and of the structure condition) the methods of proof used in 
the literature do not apply here. We will discuss the main issues along the way. 


7.1 /C-convergence and approximations of solntions of the HJB eqnation 

We first introduce the notion of /C-convergence, following[8] and [27] . 

Definition 7.1 A sequence {fn)n>o £ CtiR) is said to be K-convergent to a function f £ Cb{H) (and 
we shall write fn^f or f = K. — lim„^oo fn) if for any compact set 1C CH 

sup ||/„||oo < +00 and lim sup \f{x) - fn{x)\ = 0. 
neN 


Similarly, given / C M, a sequence {fn)n>o S Cb{I x TL) is said to be K-convergent to a function 

K, 

f £ Cb{I X R) (and we shall write again fn^f or f = K. — lim„_>oo fn) if for any compact set 1C £ R 
and for any (t,x) £ I x K. we have 

sup ||/„||oo <+00 and lim sup |/(i, x)-/„(/, x)| = 0. 
neN {t,x)&IxK 

Now we recall the definition (given in |13j . beginning of Chapter 7) of strict solution of a family of 
Kolmogorov equations. Consider the following forward Kolmogorov equation with unknown w: 

( ^ QQ* v2w(/, x) + {Ax, Vw{t, x)) -f T{t, x) t£[{),T],x£R, 

{ 2 (yq) 


w{0,x) = (j){x). 

where the functions 7": [0, T] x "H —>■ R and </> : "H —>■ R are bounded and continuous. 

Definition 7.2 By strict solution of the Kolmogorov equation |7.7[ 1 we mean a function w such that 
( w £ Cb{\0,T] X R) and w{f),x) = (j){x) 

{ w(t,-)£UCl{R),yt£[t),T]- (7.2) 

[ w(-,x) £ C'^([0,T]), Vx £ D{A) and w satisfies 

Now we prove a key approximation lemma. 
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Lemma 7.3 Let Hvvothesis \S.l\ and \3.3\ hold. Let also or lji4-35^ hold. Let v be the mild solution 

of the HJB equation h6.2i) and set w{t,x) = v(T — t,x) for {t,x) € [0,T] x H. Then there exist three 
sequences of functions i^o.n) and (Tn) such that, for all n £ N, 

4„GC'“([0,r]xK"), J-„eC'“([0,T]x7^)nET,i/2 (7.3) 

and 

io,n J^n ^ Hrainif^^w) (7-4) 

in the sense of IC-convergence. Moreover, defining (fnix) = ipnixo), £o,n(s,x) = £o,n(s,xo) and 

Wji{t,x') . — Rfcfji I Lli—s *) “t" £o,n(,^^ *)] {x)ds (7.5) 

Jo 

the following hold: 

• wn G ucl'\[o,T]xn)ni:T.,i/2, 

• Wn is a strict solution of |y. j[ ) with (fn in place of cj) and Tn + £o,n in place of T, 

• we have, in the sense of IC-convergence (the first in [0,r] x TL, the second in (0,r] xTL), 

Wn —>• W, t^^^V^Wn —>■ (7.6) 


Proof. We divide the proof in three steps. 

Step 1: choosing the three approximating sequences. We choose (fn and ^o,n to be the standard 
approximations by convolution of 4> and £o. To define Tn we observe first that, since w G then 

the function 

(t,x) 1 -^ R{t,x) := Hmini^^w{t,x)) 

has the property that there exist / : [0,r) x R" —>■ R, continuous and bounded, such that 

R{t,x) = t-^^'^f{t, {e*^x)o). 


We then let /„ be the approximation by convolution of / and define 

J'„(t, x) = t~^l'^fn{t, (e‘^a;)o). 

Step 2: proof that G UCl''^{[0,T] x TL) D 'S}p ^^2 that it is a strict solution. The fact 
that Wn G JiT.i /2 follows immediately from (17.51) . Proposition 14.Ill and Lemma 15.41 fib Differentiability 
with respect to the variable x follows arguing as in the proof of formula (I4.15|) in Lemma 14.51 indeed 
differentiating twice we get that, for all h,k G TL 

{W^Rt [f>n] {x)h, = Rt [{V^cfne^^h, (x). 

Similarly we have, for the convolution term containing LFn, 


V J Rt-s[LFnis,-)]{x),h^ 

' aiJI) ^ y J J^n(^s,z-{- + ah)'j J\f (0, Qt-s) [dz) - J Rn(^s,z -£ JC (0, Qt-s) (dz) ds 

^s, z + e^*~^^^(x + ah)^ - Tn (s, M (0, Qt-s) {dz) ds 

J J (vRn{s,z-\-{Q,Qt-s){dz)ds 


lim — 
/o Jn ^ L 


= Rts 


\7Rn{s,-),e^^ {x)ds, 

/ wJ 
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and also, arguing in the same way, 


Rt[Tn{s,-)]{x)dsh,k)'H= / ]{x)ds. 


wJ 

The convolution term involving £o,n is treated exactly in the same way. The proof that is differentiable 
with respect to time and that Wnt S i7C'b([0,r] x %) is completely analogous to what is done in [TTl 
Theorems 9.23 and 9.25]) for homogeneous Kolmogorov equations and we omit ii0. By Theorem 5.3 
in [9], see also Theorem 7.5.1 in m for Kolmogorov equations, we finally conclude that Wn is a strict 
solution to equation [731 

Step 3: proof of the convergences. First we prove that the sequences {wn) and are 

bounded uniformly with respect to n. Indeed, by and by the properties of convolutions. 


\Wnit,x)\ < \\4>n\\c 


< u\\ 


/ sup 
h yeK" 


/ sup [|J'„(s,a;)| + I4,„(s,a;)|] ds 
10 xeH 

s~^^‘^\fn{s,y)\ + |4,n(s,2/)|j ds < ll^ll 


/ sup 
Jo yGR" L 


s ^^^l/(s,y)| + |4(s,?/)| 


ds 


Moreover, using Proposition 14. 1 II and (15.91) for 4> = identity, we get 

x)\ < C'll^lloo + [ S~^^^{t - sup (1/ (s, y) \ + |4 (s, v) I) ds, 

Jo yGR" 

for a suitable C > 0. Now, with similar computations, we prove the convergences. Indeed, 

\vn{t,x) - v{t,x)\ < [ \^n {ie*^z)o + {e*^x)o) - ^ ((e‘^z)o + (e‘^a;)o) \Af{0,Qt){dz) 

Jh 

\s~^/'^Tn (s, (e®^z)o + (e‘^a;)o) - s~^l'^T (s, (e'*^z)o + (e*^x)o) 


0 JH 


+ 14,n (s, (e®^2)o + (e^^x)o) - 4 (s, (e®^2)o + (e*^a;)o) |] A/'(0, Qt-s){dz) ds. 

Since, for every compact set /C C "H the set {{e^^x)o, t G [0,T], a; € /C} is compact in M", then by the 
Dominated Convergence Theorem we get that for any compact set 1C C TL 


sup \Vn{t,x) — v{t,x)\ ^ 0. 
(i,a:)G[0,T] x/C 


(7.7) 


Moreover, using again Propositions 14. 11 1 and 15.41 fil. for a suitable C > 0, we get 
\\7^Vn{t,x) - V^v{t,x)\ 

< C y {^n ((e*^2:)o + (e‘^a;)o) - (j) {{e*^z)o + (e‘^a;)o)) Qt)idz) 

+ C f f s~^l'^]n (s, (e®^z)o + (e*^a;)o) - (s, (e®^z)o + (e*^a;)o) 

-'0 'J 'hi 


+ |4,n (s, (e®^z)o + (e*^a;)o) - 4 (s, (e®^2)o + (e*^a;)o) |) 


N{tJ,Q^_,){dz)ds. 

By Proposition 14.Ill we know that for suitable C > 0, 

I I W(0, Qt){dz) < C|| (Q?)-i/ 2 f < 

^The proof in the nonhomogeneous case can be found in the forthcoming book |16l Section 4.4]. 


c 
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Hence, applying Cauchy-Schwartz inequality we get 
\\7^Vn{t,x) - V^v{t,x)\ 

< ({e*^z)o + ie^^x)o) - K ((e*^z)o + (e‘^a;)o) |'Ar(0, Qt){dzt'^ 

JH 

+ / (t - (s, (e®^z)o + (e*^a;)o) - (s, (e®^z)o + (e*^a;)o) 

+ |4,n (s, (e'*^2;)o + (e*^x)o) - 4 (s, (e'*^2;)o + (e‘^a;)Q) |)^ A/'(0, Qt-s)idz)^ ^ . 
Applying the Dominated Convergence Theorem as for the proof of (17.71) we get the final claim 
sup \t^^^'V^Vn{t,x) — —>■ 0, for any compact set 1C C H. 

x£{0,T]xK 


□ 


Notice that, using the terminology of [S[27], the above result imply that a mild solution dZH) is 
also a KL-strong solution. In general, in an infinite dimensional Hilbert space H, existence of /C-strong 
solutions is not a routine application of the theory of evolution equations, as the operator C formally 
introduced in ib.ll is not the infinitesimal generator of a strongly continuous semigroup in the Banach space 
UCb{H). To overcome this difficulty in the already mentioned paper the theory of weakly continuous 
(or /C-continuous) semigroups has been used. 

Remark 7.4 The approximation results proved just above is needed to prove the fundamental identity, 
(see next Proposition which is the key point to get the verification theorem and the existence of 
optimal feedback controls. The idea is to apply Ito’s formula to the approximating sequence Wn composed 
with the state process Y and then to pass to the limit for n —> +oo (see e.g. or m Section f.j]). 
However in the literature the approximating sequence is taken more regular, i.e. the Wn are required to be 
classical solutions (see e.g. \13[ Section 6.2, p.l03]) of This in particular means that "S/Wn € D(A*) 

and this fact is crucial since it allows to apply Ito’s formula without requiring that the state process Y 
belongs to D{A), which would be a too strong requirement. 

In our case the used approximating procedure does not give rise in general to functions Wn with 
Vwn € D{A*). Indeed for our purposes we need that the approximants of the data 4>, Iq, P remain 
all in the space Yixp/ 2 i without this, since we only have “partial” smoothing, it is not clear at all how 
to prove the convergence of the derivative (which is needed when we pass to the limit to prove 

the fundamental identity in next subsection). Hence, in particular, since we need that, for all n G N, 
Pn G ^Ti/ 2 ’ since P is written in terms of f, we approximate f by /„, and this procedure gives the 
approximants Pn of P. In this way Pn G ^Pn ^ D{A*). 

Summing up, we are only able to find approximating strict solutions and not classical solutions. Since 
the state process Y does not belong to D(A) this fact will force us to introduce suitable regularizations Y^ 
of it (see the proof of Proposition \7.T\) . 

Remark 7.5 Calling in := ^o,n + Pn — Hminiy^Wn) is not difficult to see that the sequence Wn is a 
strict solution of the approximating HJB equation 

j dw(t^ x) ^ GG*S/^w(t, x) + {Ax, S/w{t, a:)).^ + Hniin{V^w{t, x)) + £„(t, x), t G [0, T], a; G 'H, 
[ w(0,a;) = fnix), 

This means, with the terminology used e.g. in \27j , that w is a K.-strong solution of f6.2l) . We do not go 
deeper into this since here we use the approximation only as a tool to solve our stochastic optimal control 
problem. 

Remark 7.6 In view of E.emark \S.6\ -fi) Lemma \ 7. 3\ can be easily generalized to the case when the data 
(j) and Iq are not bounded but satisfy a polynomial growth condition in the variable x as from fi3.27\ l. 

Concerning the generalization of Remark \3.(A fii} still it possible to extend the results of Lemma \ 7. 3\ 
to the case when (j) and Iq are only measurable. In this case the approximations would take place in the 
sense of the -k- convergence, which is weaker than the K-convergence and towards the K,-convergence has 
also the disadvantage of being not metrizable. For more on the notion of n-convergence the reader can 
see ]13\. Section 6.3] (see also |7^| / and U^V - 
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7.2 The Fundamental Identity and the Verification Theorem 

Now we finally go back to the control problem of Section [31 We rewrite here for the reader convenience 
the state equation (EH), 

( dy{s) = aoy{s)ds + bou{s)ds + J^^bi{^)u{s + + adWs, s £ [t,T] 

s yit) = yo, 

[ u(^) = ^ s [-c;,o), 


and its abstract reformulation (I3.15|) . 


f dY{s) = AY{s)ds + Bu{s)ds + GdWs, s £ [t,T] 
\ Y{t) = x = (a;o,a;i). 

Similarly the cost functional in (13.171) is 


J(t,a:,u) = ]E J (£o(s,3:(s)) + £i(u(s))) ds+'E(j){x{T)) 


and is rewritten as (see (13.181) 1 


J{t,x-, u) = E 



(4(s,t^(s)) + ii{u{s))) ds + Ecj){Y{T)). 


We notice that throughout this subsection and the following one, in order to avoid further technical 
difficulties, we keep the probability space P) fixed. Nothing would change if we work in the weak 

formulation, where the probability space can change (see e.g. [JS] [Chapter 2] and [13] [Chapter 2] for 
more on strong and weak formulations in finite and infinite dimension, respectively). We first prove the 
fundamental identity. 

Proposition 7.7 Let Hvvotheses \3.1\ and \3.S[ hold. Let also i4.34\ ) or {4 -‘3 5^ hold. Let v be the mild 
solution of the HJB equation id. ‘A) according to Definition \6.1l Then for every t £ [0,T] and x £ H, and 
for every admissible eontrol u, we have the fundamental identity 


j{t,x) = J{t,x;u) +E [Hrmniy^v{s,Y{s))) - Hcv{'^^v{s,Y{s))-u{s))] ds. (7.8) 


Proof. Take any admissible state-control couple (Y (•), u{-)), and let u„(t, x) := Wn{T — t, x) where {wn)n 
is the approximating sequence of strict solutions defined in Lemma l7.3l We want to apply the Ito formula 
to Vn{t,Y{t)). Unfortunately, as mentioned in Remark 17.41 this is not possible since the process Y{t) 
does not live in D{A). So we approximate it as follows. Set, for fc G N, sufficiently large, 

Yk{s;t,x) = k{k — A)~^Y{s]t,x). (7.9) 

The process Yfc is in D{A), it converges to Y (P-a.s. and s G [t,T] a.e.) and it is a strong solutioi:0 of 
the Cauchy problem 

f dYk{s) = AYk{s)ds + Bku{s)ds + GkdWs, s £ [t,T] 

( Pfc (^) — : 


where Bk = k{k — A) ^B, Gk = k{k — A) ^G and Xk = k{k — A) ^x. Now observe that, thanks to (13.7L 
([3.121) . the operator Bk is continuous, hence we can apply Dynkin’s formula (see e.g. [TH Section 1.7] or 
m Section 4.5]) to Vn{s,Yk{s)) in the interval [t,T], getting 


^;nt(s,Tfc(s)) + ^Tr GG*V\is,Yk{s)) + (Arfc(s) + BfeM(s), V?;„(s, ^(s)))^ 


EVn{Yk{T))-Vn{t,Xk) = ^J 
Using the Kolmogorov equation EU, whose strict solution is Wn, we then write 


(7.10) 


ds. 


E(j}n{Yk{T)) - Vn{t,Xk) = E [J'„(s,yfe(s)) -|-4.n(s,yfc(s)) -|- (Bfeu(s) , (s, Yfe (s))) ] ds (7.11) 

®Here we mean strong in the probabilistic sense and also in the sense of Section 5.6. 
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We first let —>■ oo in (17.111) . Since £o,n and are bounded functions and since Tn{s,x) has a 
singularity of type with respect to time and is bounded with respect to x, we can apply the 

Dominated Convergence Theorem to all terms but the last getting 


E(j)n{Y{T)) - Vn{t,X 






lim E 

k—¥-\-C)0 


{Bku(s),Vvn(s, Yk{s)))^]ds. 


(7.12) 

Concerning the last term we observe first that, by (13.71) and (13.121) we have B in the sense that, 

for all z € R" X (^([-d, 0]; M”), (writing down here C for C([-d, 0]; R”) and C* for C*([-d, 0]; R")) 


{BkU, z).^ — {BkU, ^)(r™xC*,R"xC>) ^)(R'*xC*,R"xC)) as k ^ +oo. 

This in particular imply, by the Banach-Steinhaus theorem, that {Bku}k is uniformly bounded in R” x 
C'*([—d,0];R”). Now we use the fact that Vvn{s,x) € R" x C'([—d,0];R") (see (IS.!!) ! to rewrite the 
integrand of the last term of (17.121) as 


{BkU{s),VVn{s,Yk{s)) -VVn{s,Y{s)))(^r.^C-,K’‘xC) + (s W(s))) (Rn x C* ,R" X C> 

Thanks to what said above the first term goes to 0 as fe —>■ +oo and is dominated while the second term 
is also dominated and converges to {Bu{s),'Vvn{s,Y{s))) which, thanks to (15.11) . is equal 
to (u(s), V^t!„(s, F(s)))g„, (both convergences are clearly P-a.s. and s € [t,T] a.e.). Hence 

lim e/ {Bku{s),\7vn{s,Yk{s)))^]ds = E f {u{s),V^Vn{s,Y{s)))^^]ds 

fc->.+oo 

Now we let n —>• oo. By Lemma FTSl we know that Vn{t,x) v{t,x) and (T — ty/^y^Vn{t,x) 

{T — ty^‘^V^v{t,x) pointwise. Moreover u„(f, x), (T — x) are uniformly bounded, so that, 

by dominated convergence, we get 

eJ^^ («(s), V^x„(s,F( s)))j,,„] ds ^ E^^ («(s), V^x(s, Y{s)))^J ds. 


The convergence 


E(j), 


,(x(r))-E^ [Tn{s,Y{s))+eoAs,Y{s))]ds ^E^{x{T))-E [HrmniV^v{s,Y{s)))+£o{s,Y{s))]ds 


follows directly by the construction of the approximating sequences {4>n)n iJ'n)n and (^o.n)ra- Then, 
adding and subtracting E £i(u(s))ds and letting n —?► oo in (17.121) we obtain 


v(t,x) =E</>(Y(T))+E / [4(s,T(s))+£i(M(s))]ds 


+ E^ [H^,„(V^v(s,Y(s))) - Bcv(V^v(s,Y(s));u(s))] ds 


which immediately gives the claim. 


□ 


Remark 7.8 One may wonder why we approximate the process Y with Y^ as in instead of using 

the Yosida approximants Ak of A as, e.g., in \1!A p.l44]- The reason is that we need that Y^ belongs to 
D{A), which is not guaranteed if we use Yosida approximants. A similar procedure is used, in a different 
context, in the book m, in the proof of Theorem 1.1, p. 203. 

We can now pass to prove our Verification Theorem i.e. a sufficient condition of optimality given in 
term of the mild solution v of the HJB equation (16.21) . 

Theorem 7.9 Let Huvotheses \3.1\ and \3.3\ hold. Let also or hold. Letv be the mild solution 

of the HJB equation f6‘.2l) whose existence and uniqueness is proved in i6.3\) . Then the following holds. 

• For all {t,x) € [0,T] xTL we have v(t,x) < V{t,x), where V is the value function defined in i3.22fl . 
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Let t £ [0,T] and x € 'H be fixed. If, for an admissible control u*, we have, calling Y* the 
corresponding state, 

u*{s) G a,TgmmHcv(^^v{s,Y*{s);u) 

u^U 


for a.e. s € ]P-a.s., then the pair is optimal for the control problem starting from x 

at time t and v(t,x) = V{t,x) = J{t,x;u*). 


Proof. The first statement follows directly by (17.81) due to the negativity of the integrand. Concerning 
the second statement, we immediately see that, when u = u * dLl becomes v(t,x) = J{t,x-,u*). Since 
we know that for any admissible control u 


J{t,x-,u) > V{t,x) > v{t,x), 


the claim immediately follows. 


□ 


7.3 Optimal feedback controls and v = V 

We now prove the existence of optimal feedback controls. Under the Hypotheses of Theorem 17.91 we 
define, for {s,y) G [0,T) x H, the feedback map 

T(s,y) := argmini7cy(V®w(s,y);M), (7.13) 

u£U 

where, as usual, v is the solution of the HJB equation (16.21) . Given any {t,x) G [0,T) x H, the so-called 
Closed Loop Equation (which here is, in general, an inclusion) is written, formally, as 

J dY{s) G AY{s)ds + B'Y {s,Y{s)) ds + GdWg, sG[t,T) ( 7 ta\ 

\ Y{t) = X. > 

First of all we have the following straightforward corollary whose proof is immediate from Theorem 17.91 


Corollary 7.10 Let the assumptions of Theorem \7.9\ hold true. Let v be the mild solution of Fix 

(t,x) G [0,T) X H and assume that, on [t,T) x H, the map T defined in ^7.13^ admits a measurable 
selection if : \t,T) x H ^ h. such that the Closed Loop Equation 


( dY{s) = AY{s)ds + Bif {s,Y{s)) ds + GdWs, s G [t,T) /7 i 

\Yit)=x. 

has a mild solution Y^{--,t,x) (in the sense of \11[ v.187]). Define, for s G [t,T), u.^{s) = ip{s,Y.^{s',t,x)). 
Then the couple (m^(-), U/,(-; t, cc)) is optimal at (t,x) and v(t,x) = V(t,x). If, finally, '^{t,x) is always 
a singleton and the mild solution of {7.15^ is unique, then the optimal control is unique. 

We now give sufficient conditions to verify the assumptions of Corollary 17.101 First of all define 


r(p) := {uGU : {p, u) + £i{u) = Hjnin{p)} ■ (7-16) 

Then, clearly, we have 'I'(t,a;) = r(V^u(t,x)). Observe that, under mild additional conditions on U and 
ii (for example taking U compact or ii of superlinear growth), the set T is nonempty for all p G R"*. 
If this is the case then, by [I], Theorems 8.2.10 and 8.2.11, T admits a measurable selection, i.e. there 
exists a measurable function 7 : R™ —>■ U with j(z) G r(z) for every z G R™. Since Dmin is Lipschitz 
continuous, then T, and so 7 , must be uniformly bounded. In some cases studied in the literature this 
is enough to find an optimal feedback but not in our case (read on this the subsequent Remark 17.121 
(ii)). Hence to prove existence of a mild solution of the closed loop equation (I7.15p . as requested in 
Corollary 17.101 we need more regularity of the feedback term if(s, y) = 'y{V^v{s, y)). Beyond the smooth 
assumptions on the coefficients required in Theorem 16.51 which give the regularity of V^v{t,x), we need 
the following assumption about the map T. 

Hypothesis 7.11 The set-valued map T defined in 1 7.1 d[ ) is always non empty; moreover it admits a 
Lipschitz continuous selection 7 . 
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Remark 7.12 (i) Notice that the above Hypothesis is verified if we assume that l\ : K™ —>■ R zs 

differentiable with an invertible derivative and that is Lipschitz continuous. Indeed in this 

case, see i3.23\) . the infimum of Hcv is achieved at 

u = so that ro( 2 :) = {£f)~^{z). 


(ii) The problem of the lack of regularity of the feedback law is sometimes faced (see e.g. in 

formulating the optimal control problem in the weak sense (see e.g. 123}/ or Section f.2) and 
then using Girsanov Theorem to prove existence, in the weak sense, of a mild solution of J 7.1 
when the map ^ is only measurable and bounded. This is not possible here due to the already 
mentioned absence of the structure condition in the controlled state equation (i. e. the control affects 
the system not only through the noise). 


Taking the selection 7 from Hypothesis 17.111 we consider the closed loop equation 

f dY{s) = AY{s)ds + B'j {y^v{s,Y{s))) ds + GdWs, s G [t,T] 
\ Y{t) = x = {xo,xi), 


and we have the following result. 

Theorem 7.13 Assum,e that Hyvotheses [SA[\S.Sl\3.4\ and \7. 1 l\ hold true. Fix any it,x) € [0,T)xiJ. Let 
also if.Sff) or hold. Then the closed loop equation 17.17| ) admits a unique mild solution Yy{-;t,x) 

(in the sense of m p.l87]) and setting 

%(s) = 7 {y^v{s,Y^{s;t,x))) , s G [t,T] 


we obtain an optimal control at {t,x). Moreover v{t,x) = V(t,x). 

Proof. Thanks to Corollary 17.101 it is enough to prove the existence and uniqueness of the mild solution 
of (j7.17|l . We apply a fixed point theorem to the following integral form of (17.1711 : 


y(s) = ^ e^‘^-'''>^G dWr + e'^^-'''>^B-i{V^v{r,Xr)))dr. (7.18) 

By Hypothesis 13.41 and Theorem 16.51 we get that the mild solution v of the HJB equation (16.21) is differ¬ 
entiable, with bounded derivative. Moreover, since, again by Theorem 16.51 v admits the second order 
derivative V^Vn, and = \7V^v, we deduce that •) is Lipschitz continuous, uniformly 

with respect to t. Using this Lipschitz property we can solve (17.181) by a fixed point argument. Since 
the argument to do this is straightforward we only show how to estimate the difficult term in (17.181) . We 
have 


( 7 (V^r;(r, X{r))) - jiV^v{r, Y{r)))) \ndr < ^ C'| 7 (V^i;(r, X(r))) - -f{V^v{r, Y{r)))\R^]dr 


<C \X^vir,Xir)))-X^v{r,Y{r))\Rm]dr<C r-^^^\X{r) - Y{r)\ndr 


where C is a constant that can change its values from line to line. □ 

We devote our final result to show that the identification v = V can be done, using an approximation 
procedure, also in cases when we do not know if optimal feedback controls exist. 


Theorem 7.14 Let Hvvotheses 1,7.11 \3.3\ hold. Let also Hf.Sfl l or li4-35\ ) hold. Moreover let Hypotheses 
\3.4[ (ii) and \7.ll\ hold and let (jj and £0 be uniformly continuous. Then v = V. 

Proof. We approximate £0 and by approximating £0 and (j) with standard approximants £o,n and (fn 
built by convolutions. We set 

Jn{t,x-,u) =E {£Q^ri{s,Y{s)) + £i{u{s))) ds+ E(j)n{Y{T)) (7.19) 


36 
































and call the mild solution of the HJB equation (£ is given by (16.11) 1 


dw(t, x) 


= £[w(t,-)](x) +£o,n(t,x) + Hmin(V^w(t,x)), t G [0,T], X & Ti, 


(7.20) 


dt 

w{Q,x) = (jjnix). 

In particular Wn satisfies the integral equation 

Wn{t,x) = Rt[(j)n]{x) [ Rt-s[HrninC^^Wn{s,‘)-\-iQ,n{sr)]ix) ds, t G [0,T]. X G R, (7.21) 

By Theorem 17.131 calling Vn{t,x) = Wn{T — t,x) we have 

(7.22) 


Vn{t,x) = Vn{t,x) := inf Jnit,X\u). 

u&A 


and there exists an optimal feedback control Un(s) = 'il>n(s,V(s)). Moreover, by Lemma [7.31 we know 
that 

Vn{t,x) -)> v(t,x). 

Now it is enough to prove that Vn{t,x) —>■ V{t,x) pointwise. Given e > 0, we have, for n large enough, 


Vn{t,x) = inf 
um 


E / (4,n(s,k"(s))+^i(u(s))) ds + E0„(y(T)) 


= inf 
u^lA 


E J (4(s,y(s))+£i(u(s))) ds + E^(YiT)) 

+E^' (4.n(s,r(s))-4(s,r(s))) ds + E[</.„(F(r))-</.(r(r))] 


< inf 
u£U 


E / (£o(s,F(s)) + £i(u(s))) ds + E</.(y(T)) 


+ 


where the last passage follows by the Dominated Convergence Theorem, and since 4> and io^n sxe uniformly 
continuous. We have shown that 

Vn{t, x) <V{t, x) + e. 

Exchanging the role of Vn and V we also find that the reverse inequality holds true. Hence Vn ^ V 
pointwise an the claim follows. □ 

Remark 7.15 In Theorem \ 7. 14\ we have assumed further uniform continuity on the data. When U is 
compact the result still remain true if the data are only continuous. 
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